Chapter 11 - God spiked the integers
knitr::opts_chunk$set(
fig.height = 2.5,
fig.width = 4.2,
message = FALSE,
warning = FALSE
)
library(tidyverse)
library(magrittr)
library(brms)
library(tidybayes)
library(bayesplot)
devtools::load_all()Binomial Regression
The binomial distribution is denoted \(y \sim Binomial(n, p)\). \(y\) is a count, \(p\) is the probability any particular “trial” is a success, and \(n\) is the number of trials. The come in two flavors that both use the logit link function.
- Logistic Regression is used when data is organized in to single-trial cases and the outcome variable is either 0 or 1.
- Aggregated binomial regression is used when individual trials with the same covariates are aggregated together. In this case the aggregated outcome is an integer count of successes.
Logistic regression example.
Lets prepare data from a chimpanzee social experiment. We use the recipes package to explicitly create less than full rank dummy variables.
library(recipes)
library(rethinking)
data(chimpanzees)
d <- as_data_frame(chimpanzees) %>%
select(pulled_left, prosoc_left, condition, actor)
rm(chimpanzees)
detach("package:rethinking")
d <- d %>%
mutate(
treatment = 1 + prosoc_left + 2 * condition,
treatment = case_when(
treatment == 1 ~ "R/N",
treatment == 2 ~ "L/N",
treatment == 3 ~ "R/P",
treatment == 4 ~ "L/P"),
actor = as.factor(actor))
df <- recipe(pulled_left ~ 0 + treatment + actor, data = d) %>%
step_dummy(treatment, actor, one_hot = TRUE) %>%
prep() %>%
bake(new_data = d)
df %>%
skimr::skim()Skim summary statistics
n obs: 504
n variables: 12
── Variable type:integer ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
variable missing complete n mean sd p0 p25 p50 p75 p100 hist
pulled_left 0 504 504 0.58 0.49 0 0 1 1 1 ▆▁▁▁▁▁▁▇
── Variable type:numeric ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
variable missing complete n mean sd p0 p25 p50 p75 p100 hist
actor_X1 0 504 504 0.14 0.35 0 0 0 0 1 ▇▁▁▁▁▁▁▁
actor_X2 0 504 504 0.14 0.35 0 0 0 0 1 ▇▁▁▁▁▁▁▁
actor_X3 0 504 504 0.14 0.35 0 0 0 0 1 ▇▁▁▁▁▁▁▁
actor_X4 0 504 504 0.14 0.35 0 0 0 0 1 ▇▁▁▁▁▁▁▁
actor_X5 0 504 504 0.14 0.35 0 0 0 0 1 ▇▁▁▁▁▁▁▁
actor_X6 0 504 504 0.14 0.35 0 0 0 0 1 ▇▁▁▁▁▁▁▁
actor_X7 0 504 504 0.14 0.35 0 0 0 0 1 ▇▁▁▁▁▁▁▁
treatment_L.N 0 504 504 0.25 0.43 0 0 0 0.25 1 ▇▁▁▁▁▁▁▂
treatment_L.P 0 504 504 0.25 0.43 0 0 0 0.25 1 ▇▁▁▁▁▁▁▂
treatment_R.N 0 504 504 0.25 0.43 0 0 0 0.25 1 ▇▁▁▁▁▁▁▂
treatment_R.P 0 504 504 0.25 0.43 0 0 0 0.25 1 ▇▁▁▁▁▁▁▂Remember, our coefficients ultimately need to be interpreted in the space on the other side of the link function. What would a flat prior that we used in the gaussian world, like \(normal(0, 10)\), mean in the logistic world? Well too much of that density would be piled up outside the bounds of probability making anything but 1 and 0 probable. A flat prior in the logit space is not a flat prior in the outcome probability space.
rnorm(1e4, 0, 10) %>%
inv_logit_scaled() %>%
tibble::enframe() %>%
ggplot(aes(value)) +
geom_density(fill = palette[7], colour = "transparent") +
theme_burgyl() +
ggtitle("Improper prior - normal(0, 10)",
"Priors need to be callibrated to the outcome probability space")
Lets try something more reasonable.
rnorm(1e4, 0, 1.5) %>%
inv_logit_scaled() %>%
tibble::enframe() %>%
ggplot(aes(value)) +
geom_density(fill = palette[7], colour = "transparent") +
theme_burgyl() +
ggtitle("A more proper prior", "normal(0, 1.5)")
Now we fit the data with a parameter for each actor and each possible treatment.
b10.1 <-
brm(data = df, family = bernoulli,
pulled_left ~ 0 + .,
prior = prior(normal(0, 1.5), class = b),
refresh = 0, cores = 4)
summary(b10.1) Family: bernoulli
Links: mu = logit
Formula: pulled_left ~ 0 + (treatment_L.N + treatment_L.P + treatment_R.N + treatment_R.P + actor_X1 + actor_X2 + actor_X3 + actor_X4 + actor_X5 + actor_X6 + actor_X7)
Data: df (Number of observations: 504)
Samples: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
total post-warmup samples = 4000
Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
treatment_L.N 0.88 0.50 -0.12 1.85 479 1.00
treatment_L.P 0.74 0.50 -0.22 1.71 440 1.00
treatment_R.N 0.28 0.50 -0.73 1.22 447 1.00
treatment_R.P -0.13 0.50 -1.13 0.82 448 1.00
actor_X1 -0.78 0.50 -1.76 0.21 456 1.00
actor_X2 3.66 0.82 2.18 5.48 1146 1.00
actor_X3 -1.08 0.52 -2.06 -0.04 465 1.00
actor_X4 -1.08 0.52 -2.09 -0.06 535 1.00
actor_X5 -0.78 0.51 -1.77 0.26 492 1.00
actor_X6 0.15 0.51 -0.83 1.18 475 1.00
actor_X7 1.66 0.57 0.59 2.82 582 1.00
Samples were drawn using sampling(NUTS). For each parameter, Eff.Sample
is a crude measure of effective sample size, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).Below we examine our coefficients. We need to convert each parameter to the outcome scale. We can do this with brms::inv_logit_scaled. Note how antisocial actor 2 is.
post <- posterior_samples(b10.1) %>%
gather(parameter, value) %>%
mutate(value = inv_logit_scaled(value))
p_actors <- post %>%
filter(str_detect(parameter, "b_actor.*")) %>%
ggplot(aes(y=parameter, x=value)) +
tidybayes::geom_halfeyeh(fill = palette[3]) +
theme_burgyl()
p_treat <- post %>%
filter(str_detect(parameter, "b_treat.*")) %>%
ggplot(aes(y=parameter, x = value)) +
geom_halfeyeh(fill = palette[5]) +
theme_burgyl()
gridExtra::grid.arrange(p_actors, p_treat, ncol = 2)
We can now compare the constrasts in treatments.
posterior_samples(b10.1) %>%
select(starts_with("b_treat")) %>%
transmute(
` left lever` = (b_treatment_L.N - b_treatment_L.P),
` right lever` = (b_treatment_R.N - b_treatment_R.P)) %>%
gather(parameter, value) %>%
ggplot(aes(y = parameter, fill = parameter, x = value)) +
geom_halfeyeh(.width = c(.5, .95)) +
geom_vline(xintercept = 0, colour = alpha(palette[7], .6), linetype = 2) +
theme_burgyl() +
scale_fill_manual(values = c(palette[3], palette[5])) +
scale_x_continuous(breaks = round(seq(from=-.6, to = 1.4, by = .2),1)) +
theme(legend.title = element_blank(),
axis.text.y = element_blank(),
axis.ticks.y = element_blank()) +
ylab("") + xlab("Log odds of pulling the left lever") +
ggtitle("Prosocial behavior is not statistically distinguishable")
Aggregated binomial
When that same data is aggregated, we can switch to a binomial regression instead of a bernouli. Note the use of the criteria | trials() syntax.
library(rethinking)
data(chimpanzees)
d <- chimpanzees
detach("package:rethinking")
d_aggregated <- d %>%
select(-recipient, -block, -trial, -chose_prosoc) %>%
group_by(actor, condition, prosoc_left) %>%
summarise(x = sum(pulled_left))
d_aggregated %>%
filter(actor %in% c(1, 2))loo is misleading in the aggregate case, specifically because you are leaving out aggregations at a time. Stick to the bernoulli if you want to use loo.
b10.5 <-
brm(data = d_aggregated, family = binomial,
x | trials(18) ~ 1 + prosoc_left + condition:prosoc_left ,
prior = c(prior(normal(0, 10), class = Intercept),
prior(normal(0, 10), class = b)),
iter = 2500, warmup = 500, cores = 2, chains = 2,
refresh = 0)
loo(b10.5)
Computed from 4000 by 28 log-likelihood matrix------
Monte Carlo SE of elpd_loo is 0.1.
Pareto k diagnostic values:
Count Pct. Min. n_eff
(-Inf, 0.5] (good) 27 96.4% 489
(0.5, 0.7] (ok) 1 3.6% 687
(0.7, 1] (bad) 0 0.0% <NA>
(1, Inf) (very bad) 0 0.0% <NA>
All Pareto k estimates are ok (k < 0.7).
See help('pareto-k-diagnostic') for details.Graduate school admissions
Each row is a department-gender with the number of acceptances out of total applications. This is just an aggregated bernoulli that we will represent as independent binomial trials to collect the effect of gender.
library(rethinking)
data(UCBadmit)
d <- UCBadmit
detach(package:rethinking)
library(brms)
rm(UCBadmit)
d <-
d %>%
mutate(
male = ifelse(applicant.gender == "male", 1, 0),
female = -1*male + 1)
d %>%
glimpseObservations: 12
Variables: 7
$ dept <fct> A, A, B, B, C, C, D, D, E, E, F, F
$ applicant.gender <fct> male, female, male, female, male, female, male, female, male, female, male, female
$ admit <int> 512, 89, 353, 17, 120, 202, 138, 131, 53, 94, 22, 24
$ reject <int> 313, 19, 207, 8, 205, 391, 279, 244, 138, 299, 351, 317
$ applications <int> 825, 108, 560, 25, 325, 593, 417, 375, 191, 393, 373, 341
$ male <dbl> 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0
$ female <dbl> 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1Here is the model we are running. We use a relatively flat prior on the outcome scale. \[ admission \sim Binomial(N_i, p_i) \\ logit(p_i) = \alpha * male + \beta * female \\ \alpha, \beta \sim Normal(0, 1.5) \]
b10.6 <-
brm(data = d, family = binomial,
admit | trials(applications) ~ 0 + male + female,
prior = c(prior(normal(0, .15), class = b)),
iter = 2500, warmup = 500, cores = 4, chains = 4,
refresh = 0, sample_prior = "yes")
plot(b10.6)
prior_samples(b10.6) %>%
gather(parameter) %>%
ggplot(aes(y = parameter, x = inv_logit_scaled(value))) +
geom_halfeyeh(.width = c(.95, .5),
fill = palette[5],
point_interval = median_hdi) +
theme_burgyl() +
ggtitle("Weakly regularizing prior for coefficients",
"95% and 50% highest density interval about the median") 
posterior_samples(b10.6) %>%
select(starts_with("b_")) %>%
gather(parameter) %>%
ggplot(aes(value, parameter, fill = parameter)) +
geom_halfeyeh() +
theme_burgyl("tl") + ylab("") +
scale_fill_manual(values = c(palette[3], palette[6])) +
theme(legend.title = element_blank(),
axis.ticks.y = element_blank(),
axis.text.y = element_blank())
Now lets compute the contrasts.
posterior_samples(b10.6) %>%
transmute(
`logit scale` = b_male - b_female,
`outcome scale` = inv_logit_scaled(b_male) - inv_logit_scaled(b_female)) %>%
gather(parameter) %>%
ggplot(aes(y = parameter, x = value, fill = parameter)) +
geom_halfeyeh(point_interval = median_hdi,
.width = c(.95, .5)) +
ggtitle("Men have a 12% higher acceptance rate than women",
"95% and 50% highest density interval about the median") +
theme_burgyl("tl") + ylab("") +
scale_fill_manual(values = c(palette[3], palette[6])) +
theme_burgyl() +
theme(axis.text.y = element_blank(),
axis.ticks.y = element_blank())
I personally find that the interpretation is more interpretable on the outcome scale.
spread_draws(b10.6, b_male, b_female) %>%
transmute(diff = inv_logit_scaled(b_male) - inv_logit_scaled(b_female)) %>%
median_hdi()What did we just do here? Recall that this our formula for a logistic regression. \[ logit(p) = -.2062730 * is\_male + -.7469153 * is\_female \]
For each gender, we predict the following probabilities. That 12 is the difference in percentage points! Not that males are 12% mroe likely to get admitted!
spread_draws(b10.6, b_male, b_female) %>%
select(starts_with("b_")) %>%
gather(parameter, value) %>%
mutate(value = inv_logit_scaled(value)) %>%
ggplot(aes(y = parameter, x = value)) +
tidybayes::stat_intervalh(point_interval = median_hdi)
Finally, lets perform a posterior prediction check. It turns out we are pretty off.
The problem in this case is that males and females do not apply to the same departments, and departments vary in their rates of admission. This makes the answer misleading. You can see the steady decline in admission probability for both males and females from department A to department F. Females in these data tended not to apply to departments like A and B, which had high overall admission rates. Instead they applied in large numbers to departments like F, which admitted less than 10% of applicants. So while it is true overall that females had a lower probability of admission in these data, it is clearly not true within most departments.
d %>%
mutate(prediction = predict(b10.6)[,1],
residuals = admit - prediction) %>%
group_by(dept) %>%
summarise(resid = mean(residuals)) %>%
arrange(desc(abs(resid)))What we will now do is include variables for each department. That way the gender variables will only provide marginal information on top of the baseline acceptance rates of each department.
summary(b10.8)
Family: binomial
Links: mu = logit
Formula: admit | trials(applications) ~ 0 + male + female + dept_A + dept_B + dept_C + dept_D + dept_E + dept_F
Data: df (Number of observations: 12)
Samples: 4 chains, each with iter = 2500; warmup = 500; thin = 1;
total post-warmup samples = 8000
Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
male -0.35 0.06 -0.48 -0.22 3106 1.00
female -0.43 0.07 -0.57 -0.29 3698 1.00
dept_A 0.79 0.08 0.64 0.95 4189 1.00
dept_B 0.68 0.09 0.51 0.85 5141 1.00
dept_C -0.18 0.08 -0.34 -0.03 4596 1.00
dept_D -0.22 0.08 -0.38 -0.06 4559 1.00
dept_E -0.50 0.09 -0.68 -0.32 5368 1.00
dept_F -1.36 0.09 -1.55 -1.17 5103 1.00
Samples were drawn using sampling(NUTS). For each parameter, Eff.Sample
is a crude measure of effective sample size, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).Lets compute the contrast again. The difference in acceptance rates is no longer significant. What happened?
posterior_samples(b10.8) %>%
transmute(male = inv_logit_scaled(b_male),
female = inv_logit_scaled(b_female),
diff = male - female) %>%
select(diff) %>% gather(parameter) %>%
ggplot(aes(y=parameter, x=value)) +
geom_eyeh(fill = palette[6],
.width = c(.5, .95)) +
geom_vline(xintercept = 0, linetype = 2) +
theme_burgyl()
Department is a confound. It opens up a backdoor pipe to acceptance rates.
library(dagitty)
dag <- dagitty("dag {
gender -> acceptance
department -> acceptance
gender -> department
}")
plot(dagitty::graphLayout(dag))
Multinomial and Categorical Models
When more than two types of unordered events are possible, and the probability of each type of event is constant across trials, then the maximum entropy distribution is the multinomial distribution. Modeling this type of distribution can be done by either using a generalization of the logit link or by transforming the multinomial likelihood into a series of Poisson likelihoods.
The multinomial logit link function takes a vector of scores, one for each event type, and computes the probability of a specific event, k.
\[ Pr(k|s_1, s_2, \dots, s_K) = \frac {exp(s_k)} {\sum^K_{i=1}exp(s_i)} \]
In a multinomial GLM, you need to build K-1 linear models for K events, each of which can (1) use any predictor parameters it would like and (2) get assigned any predictor values it wants.
Example: Predicting career choice as a function of income. Here \(\beta_{INCOME}\) appears in each linear model for each possible career choice.
N <- 100
# simulate family incomes for each individual
family_income <- runif(N)
# assign a unique coefficient for each type of event
b <- (1:-1)
career <- rep(NA, N) # empty vector of choices for each individual
for ( i in 1:N ) {
score <- 0.5*(1:3) + b*family_income[i]
p <- softmax(score[1],score[2],score[3])
career[i] <- sample( 1:3 , size=1 , prob=p )
}
data_frame(
career,
family_income
)Alternatively, you can specify brand new parameters for your linear models. Interpretation is very difficult for models like these.
m10.17 <- map(
alist(
career ~ dcategorical(softmax(0,s2,s3)),
s2 <- a2 + b2*family_income,
s3 <- a3 + b3*family_income,
c(a2, a3, b2, b3) ~ dnorm(0,5)
), data=list(career=career,family_income=family_income))
m10.17
Quadratic approximate posterior distribution
Formula:
career ~ dcategorical(softmax(0, s2, s3))
s2 <- a2 + b2 * family_income
s3 <- a3 + b3 * family_income
c(a2, a3, b2, b3) ~ dnorm(0, 5)
Posterior means:
a2 a3 b2 b3
-0.03633395 0.55688952 -0.52983875 -1.63672335
Log-likelihood: -106.93 Poisson regression
Often times, the upper limit of your binomial, \(n\), is unknown. In these cases you’d want to use a poisson distriution. Recall that a binomial distributions mean is \(Np\) and its variance is \(Np(1-p)\). When N is very large and p is very small then these two are approximately the same.
y <- rbinom(1e5, 1000, 1/1000)
tibble(
mean = round(mean(y), 2),
var = round(var(y), 2))This unified parameter is represented by the poisson distribution’s sole parameter \(\lambda\) which is the expected value of the outcome y. It is also the expected variance of the counts of y. The conventional link function for a Poisson model is the log link, which ensures our count is always positive. \[ y_i \sim Poisson(\lambda_i) \\ log(\lambda_i) + \alpha + \beta(x_i - \bar x) \] The log link also implies an exponential relationship between predictors and the expected value.
Example: Oceanic tool complexity
Here we examine how population size determines complexity of tools developed.
suppressPackageStartupMessages(library(rethinking))
data(Kline)
d <- as_data_frame(Kline)
rm(Kline)
detach("package:rethinking")
df <- d %>%
mutate(
log_pop = scale(log(population), center = TRUE),
contact_high = ifelse(contact == "high", 1, 0),
contact_low = ifelse(contact == "low", 1, 0))
df %>%
select(culture, population, log_pop,
contact_high, contact_low, total_tools) %>%
skimr::skim()Skim summary statistics
n obs: 10
n variables: 6
── Variable type:character ──────────────────────────────────────────────────────────────────────────────
variable missing complete n min max empty n_unique
log_pop 0 10 10 15 19 0 10
── Variable type:factor ─────────────────────────────────────────────────────────────────────────────────
variable missing complete n n_unique top_counts ordered
culture 0 10 10 10 Chu: 1, Haw: 1, Lau: 1, Mal: 1 FALSE
── Variable type:integer ────────────────────────────────────────────────────────────────────────────────
variable missing complete n mean sd p0 p25 p50 p75 p100 hist
population 0 10 10 34109.1 84793.03 1100 3897.75 7700 12050 275000 ▇▁▁▁▁▁▁▁
total_tools 0 10 10 34.8 17.85 13 22.5 30.5 42.25 71 ▇▇▇▃▃▃▁▃
── Variable type:numeric ────────────────────────────────────────────────────────────────────────────────
variable missing complete n mean sd p0 p25 p50 p75 p100 hist
contact_high 0 10 10 0.5 0.53 0 0 0.5 1 1 ▇▁▁▁▁▁▁▇
contact_low 0 10 10 0.5 0.53 0 0 0.5 1 1 ▇▁▁▁▁▁▁▇Here is the model we’ll fit: \[ total\_tools \sim Poisson(\lambda_i) \\ log(\lambda_i) = \alpha_{contact{[i]}} + \beta_{contact[i]}*log(population_i) \\ \alpha_j \sim normal(3, .5) \\ \beta_j \sim normal(0, .2) \]
We chose our priors to make sure that they are sensible in the outcome space. The purpose of our more spiked beta prior is to dissuade our model from believing explosive exponential relationships.
tibble(
alpha = rnorm(1e5, 3, .5),
beta = rnorm(1e5, 0, .2)
) %>%
gather %>%
mutate(value = exp(value)) %>%
filter(!is.na(value)) %>%
ggplot(aes(y=key, x=value)) +
geom_halfeyeh(fill = palette[5]) +
coord_cartesian(xlim = c(0, 50)) +
theme_burgyl() + ylab("Prior") + xlab("exp(prior)")
b10.9 <-
brm(data = df, family = poisson,
total_tools ~ 1,
prior = c(prior(normal(3, .5), class = Intercept)),
chains = 4, cores = 4, sample_prior = "yes", refresh = 0)Compiling the C++ model
Start samplingb10.9 Family: poisson
Links: mu = log
Formula: total_tools ~ 1
Data: df (Number of observations: 10)
Samples: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
total post-warmup samples = 4000
Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
Intercept 3.54 0.05 3.43 3.65 1678 1.00
Samples were drawn using sampling(NUTS). For each parameter, Eff.Sample
is a crude measure of effective sample size, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).df <- df %>%
mutate(log_pop = log(population))
b10.10 <-
brm(data = df, family = poisson,
total_tools ~ 0 + contact_low + contact_high + log_pop + contact_high:log_pop,
prior = c(prior(normal(0, 1), class = b)),
chains = 4, cores = 4, refresh = 0, sample_prior = "yes")Compiling the C++ model
Start samplingb10.10 Family: poisson
Links: mu = log
Formula: total_tools ~ 0 + contact_low + contact_high + log_pop + contact_high:log_pop
Data: df (Number of observations: 10)
Samples: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
total post-warmup samples = 4000
Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
contact_low 0.83 0.34 0.17 1.47 1268 1.00
contact_high 0.22 0.81 -1.37 1.77 1494 1.00
log_pop 0.27 0.03 0.21 0.34 1298 1.00
contact_high:log_pop 0.10 0.09 -0.08 0.29 1386 1.00
Samples were drawn using sampling(NUTS). For each parameter, Eff.Sample
is a crude measure of effective sample size, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).loo.10.10 <- loo(b10.10, save_psis = TRUE, cores = 2)Found 1 observations with a pareto_k > 0.7 in model 'b10.10'. It is recommended to set 'reloo = TRUE' in order to calculate the ELPD without the assumption that these observations are negligible. This will refit the model 1 times to compute the ELPDs for the problematic observations directly.print(loo.10.10)
Computed from 4000 by 10 log-likelihood matrix------
Monte Carlo SE of elpd_loo is NA.
Pareto k diagnostic values:
Count Pct. Min. n_eff
(-Inf, 0.5] (good) 9 90.0% 486
(0.5, 0.7] (ok) 0 0.0% <NA>
(0.7, 1] (bad) 1 10.0% 155
(1, Inf) (very bad) 0 0.0% <NA>
See help('pareto-k-diagnostic') for details.plot(loo.10.10, label_points = TRUE)
td <- tibble(
population = seq(from = 0, to = 275000, length.out = 250)) %>%
expand(nesting(population), contact = c("low", "high")) %>%
mutate(
log_pop = log(population),
contact_high = ifelse(contact == "high", 1, 0),
contact_low = ifelse(contact == "low", 0, 1))
td <- td %>%
bind_cols(
as_data_frame(
predict(b10.10, newdata = td))) %>%
filter(!is.nan(Estimate))NAs producedNAs producedp1 <- td %>%
ggplot(aes(x = population, y = Estimate,
ymin = `Q2.5`, ymax = `Q97.5`,
group = contact)) +
geom_ribbon(colour = "transparent", alpha = .6,
aes(fill = contact)) +
geom_line() +
geom_point(data = d, inherit.aes = FALSE,
mapping = aes(x = population, y = total_tools,
shape = contact, colour = contact)) +
theme_burgyl("tl") + ylab("Predicted Total Tools") +
scale_fill_manual(values = c(palette[3], palette[5])) +
scale_color_manual(values = c(palette[6], palette[7])) #+
# ggtitle("Posterior prediction plot",
# "total_tools ~ contact_low + contact_high + log_pop + contact_high:log_pop")
p2 <- td %>%
ggplot(aes(x = log_pop, y = Estimate,
ymin = `Q2.5`, ymax = `Q97.5`,
group = contact)) +
geom_ribbon(colour = "transparent", alpha = .6,
aes(fill = contact)) +
geom_line() +
geom_point(data = d, inherit.aes = FALSE,
mapping = aes(x = log(population), y = total_tools,
shape = contact, colour = contact)) +
theme_burgyl() + ylab("") + xlab("log(population)") +
theme(legend.position="none") +
scale_fill_manual(values = c(palette[3], palette[5])) +
scale_color_manual(values = c(palette[6], palette[7]))
gridExtra::grid.arrange(p1, p2, ncol = 2,
top = "Posterior prediction plot")
Example: Exposure and the offset
The rate parameter, \(\lambda\), of a Poisson distribution can be normalized across different exposure frequencies. McElreath gives the example of combining data from a daily log and a weekly log.
\[ y_i \sim Poisson(\lambda_i) \\ log(\lambda_i) = log(\frac{\mu_i}{\tau_i}) = \alpha + \beta x_i \\ log(\mu_i) = log(\tau_i) + \alpha + \beta x_i \]
\(\tau\) is just a column of data, often referred to as the offset. Lets simulate an example to show how this works where one monestary records its manuscript completions daily while another does so weekly.
num_days <- 30
num_weeks <- 4
d <- data_frame(
manuscripts = c(rpois(num_days, 1.5),
rpois(num_weeks, .5 * 7)),
num_days = c(rep(1, 30), rep(7, 4)),
monastery_id = c(rep(0, 30), rep(1, 4)))
d <- d %>%
mutate(
log_num_days = log(num_days),
monastary_1 = as.numeric(monastery_id == 0),
monastary_2 = as.numeric(monastery_id == 1))
d %>%
tail(10)Now lets fit the model.
b10.15 <-
brm(data = d, family = poisson,
manuscripts ~ 0 + offset(log_num_days) + monastary_1 + monastary_2,
prior = c(prior(normal(0, 1), class = b)),
sample_prior = "yes", refresh = 0,
iter = 2500, warmup = 500, cores = 2, chains = 2)Compiling the C++ model
recompiling to avoid crashing R session
Start samplingb10.15 Family: poisson
Links: mu = log
Formula: manuscripts ~ 0 + offset(log_num_days) + monastary_1 + monastary_2
Data: d (Number of observations: 34)
Samples: 2 chains, each with iter = 2500; warmup = 500; thin = 1;
total post-warmup samples = 4000
Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
monastary_1 0.32 0.15 0.01 0.61 2527 1.00
monastary_2 -0.56 0.24 -1.06 -0.12 2753 1.00
Samples were drawn using sampling(NUTS). For each parameter, Eff.Sample
is a crude measure of effective sample size, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).rates <- fixef(b10.15) %>%
as_data_frame %>%
pull(Estimate) %>%
exp
subtitle <- glue("Monastary 2 produces manuscripts {round(rates[1] / rates[2], 1)} times as fast as monastary 1")
posterior_samples(b10.15) %>%
select(starts_with("b_")) %>%
transmute(rate_diff = exp(b_monastary_1) / exp(b_monastary_2)) %>%
gather(Parameter, Value) %>%
ggplot(aes(y = Parameter, x = Value)) +
geom_vline(xintercept = rates[1] / rates[2]) +
tidybayes::stat_intervalh(.width = c(.9, .95, .99)) +
geom_vline(xintercept = 0, linetype = 2) +
coord_cartesian(xlim = c(0, 6)) +
theme_bw() + theme(panel.grid = element_blank(),
axis.text.y = element_blank(),
axis.ticks.y = element_blank(),
legend.position = c(1, 1),
legend.justification = c(1, 1),
legend.background = element_rect(fill = "transparent")) +
ylab("") + xlab("Times increase in manuscript creation rate") +
annotate("text", x = 3.1, y = 1.4,
label = glue::glue(
"Median HDPI: {round(rates[1] / rates[2], 1)}")) +
ggtitle("Monastary 2 creates manuscripts significantly faster than Monastary 1")
Multinomial in disguise as Poisson
suppressPackageStartupMessages(library(rethinking))
data(UCBadmit)
df <- as_data_frame(UCBadmit)
detach("package:rethinking")
dfb.binom <- brm(admit | trials(applications) ~ 1,
df, binomial,
prior = prior(normal(0, 100), class = "Intercept"),
refresh = 0)
b.binom Family: binomial
Links: mu = logit
Formula: admit | trials(applications) ~ 1
Data: df (Number of observations: 12)
Samples: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
total post-warmup samples = 4000
Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
Intercept -0.46 0.03 -0.52 -0.40 1387 1.00
Samples were drawn using sampling(NUTS). For each parameter, Eff.Sample
is a crude measure of effective sample size, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).summary(m_pois)
Inference for Stan model: 62dbe67ebbb91aa25dccdb88353bb466.
3 chains, each with iter=2000; warmup=1000; thin=1;
post-warmup draws per chain=1000, total post-warmup draws=3000.
mean se_mean sd 2.5% 25% 50% 75% 97.5% n_eff Rhat
a1 4.99 0.00 0.02 4.94 4.97 4.99 5.00 5.03 2100 1
a2 5.44 0.00 0.02 5.40 5.43 5.44 5.46 5.48 2208 1
lp__ 19302.17 0.03 0.97 19299.64 19301.76 19302.45 19302.89 19303.14 1208 1
Samples were drawn using NUTS(diag_e) at Sat Feb 16 16:07:48 2019.
For each parameter, n_eff is a crude measure of effective sample size,
and Rhat is the potential scale reduction factor on split chains (at
convergence, Rhat=1).k <- as.numeric(coef(m_pois))
tibble(
binom_fit = inv_logit_scaled(fixef(b.binom))[,1],
pois_fit = exp(k[1]) / (exp(k[1]) + exp(k[2])))NACensoring and survival
Survival models measure displacement, which are continuous deviations from some point of reference. Examples of this are durations (time to discharge). When all we know is the average displacement, the exponential disgribution is the maximum entropy distribution. Alternatively, the gamma distribution is the maximum entropy distribution for fixed mean value and fixed mean magnitude.
Suvival models also deal with censoring, which is when some other event happens and gets in the way of measuring the event of interest (death instead of discharge).
Cats example
Lets examine cats in Austin where we are interested in the time until their adoption. We have one entry per cat in the data set.
suppressPackageStartupMessages(library(rethinking))
data(AustinCats)
d <- as_data_frame(AustinCats) %>%
mutate(adopt = ifelse(out_event == "Adoption", 1L, 0L))
rm(AustinCats)
detach("package:rethinking")
(d %>%
transmute(id,
adopt,
days_to_event,
is_black = as.integer(color=="Black")) ->
d)We’ll run the following model to predict the number of days. For adopted cats, the days to adoption is exponentially distributed and easy to encode. For non-adopted cats, we employ the cumulative probability distribution which gives the proportion of cats adopted before or at a certain number of days. Therefore, \(1 - cumulative\_distribution\) gives the proportion of cats not yet adopted at day \(d\). This is called the complementary cumulative probability distribution and is displayed in the second line below.
\[ D_i | A_i = 1 \sim Exponential(\lambda_i) \\ D_i | A_i = 0 \sim ExponentialCCDF(\lambda_i) \\ \lambda_i = \frac{1}{\mu_i} \\ log \mu_i = \alpha_{CID[i]} \]
Lets run this in STAN.
post <- as_data_frame(extract.samples(m11.14)$a) %>%
rename(black = V1,
other = V2)
post %>%
gather(colour) %>%
mutate(value = exp(value)) %>%
ggplot(aes(x = value, y = colour)) +
geom_halfeyeh(.width = c(.95, .5)) +
theme(panel.background = element_rect(fill = "transparent",
colour = "black")) +
xlab("Days to adoption") + ylab("Cat colour")
Homework
10H3
First lets gather and preprocess the data.
library(MASS)
data(eagles)
df <- as_data_frame(eagles)
rm(eagles)
detach("package:MASS")
df <- df %>%
transmute(
successes = y,
attempts = n,
pirate_size = ifelse(P == 'L', "large", "small"),
adult = ifelse(A == "I", "immature", "adult"),
victim_size = ifelse(V == "L", "large", "small"))
library(recipes)
df <- df %>%
recipe(successes ~ .) %>%
step_dummy(pirate_size, adult, victim_size, one_hot = TRUE) %>%
prep %>%
juice
dfWe drop the intercept and include all factor levels of our binary variables.
hypothesis <- bf(successes | trials(attempts) ~ 0 + .)
get_prior(hypothesis, family = binomial, data = df)Our stan estimates look solid so lets just run with those.
fit.10h3 <- brm(family = binomial, data = df,
formula = hypothesis,
prior = c(prior(normal(0, 4), class = b)),
chains = 4, cores = 4, refresh = 0)
plot(fit.10h3)
Lets take a look at our estimates on the outcome scale.
posterior_samples(fit.10h3) %>%
select(starts_with("b_")) %>%
gather(parameter, value) %>%
mutate(value = inv_logit_scaled(value)) %>%
ggplot(aes(y=parameter, x = value)) +
geom_halfeyeh(.width = c(.5, .95))
as_data_frame(predict(fit.10h3, newdata = df)) %>%
bind_cols(index=1:nrow(df), df, .) %>%
ggplot(aes(x = index, colour = as.factor(pirate_size_large), shape = as.character(adult_adult))) +
geom_crossbar(aes(y = Estimate/attempts, ymin = `Q2.5`/attempts, ymax = `Q97.5`/attempts)) +
geom_point(aes(y = successes/attempts), size = 3) +
theme(panel.grid = element_blank())
NA
NA
NAfit.10h3.2 <- update(fit.10h3, refresh = 0,
formula = bf(successes | trials(attempts) ~ 1 + adult_adult +
pirate_size_large +
victim_size_large +
adult_adult:pirate_size_large))
fit.10h3.2 Family: binomial
Links: mu = logit
Formula: successes | trials(attempts) ~ adult_adult + pirate_size_large + victim_size_large + adult_adult:pirate_size_large
Data: df (Number of observations: 8)
Samples: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
total post-warmup samples = 4000
Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
Intercept -0.60 1.01 -2.76 1.25 1940 1.00
adult_adult 3.08 1.17 0.90 5.44 1620 1.00
pirate_size_large 6.06 1.32 3.64 8.84 1561 1.00
victim_size_large -4.96 1.06 -7.30 -3.12 1790 1.00
adult_adult:pirate_size_large -2.59 1.29 -5.18 -0.08 1634 1.00
Samples were drawn using sampling(NUTS). For each parameter, Eff.Sample
is a crude measure of effective sample size, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).plot(fit.10h3.2)
waic(fit.10h3, fit.10h3.2) WAIC SE
fit.10h3 30.95 8.43
fit.10h3.2 21.33 5.91
fit.10h3 - fit.10h3.2 9.62 3.28loo(fit.10h3, fit.10h3.2, reloo = TRUE) LOOIC SE
fit.10h3 42.69 13.22
fit.10h3.2 28.23 8.83
fit.10h3 - fit.10h3.2 14.46 4.7410H.4
First lets load the data and take a look at it.
library(rethinking)
data(salamanders)
df <- as_data_frame(salamanders)
rm(salamanders)
detach("package:rethinking")
df <- df %>%
transmute(
site = SITE,
num_salaman = SALAMAN,
pct_cover = PCTCOVER,
age_trees = FORESTAGE)
df %>% skimr::skim()Skim summary statistics
n obs: 47
n variables: 4
── Variable type:integer ─────────────────────────────────────────────────────────────────────────────
variable missing complete n mean sd p0 p25 p50 p75 p100 hist
age_trees 0 47 47 168.81 193.51 0 29.5 64 266.5 675 ▇▁▂▁▁▁▁▁
num_salaman 0 47 47 2.47 3.37 0 0 1 3 13 ▇▂▁▁▁▁▁▁
pct_cover 0 47 47 58.98 35.78 0 18 83 88 93 ▃▁▁▁▁▁▁▇
site 0 47 47 24 13.71 1 12.5 24 35.5 47 ▇▇▇▇▇▇▇▇Now for some preprocessing. We want to try a few different transformations of our predictors.
dm <- df %>%
mutate(log_age_trees = log(age_trees)) %>%
filter(is.finite(log_age_trees)) %>%
mutate(
pct_cover_cs = scale(pct_cover),
log_age_trees_cs = scale(log_age_trees),
age_trees_cs = scale(age_trees))
dm %>% headInterestingly, the best performing model only uses the percent of coverage. It appears the age of trees adds too much noise and does not generalize well.
fit.10h.4.1 <- brm(num_salaman ~ 1, refresh = 0,
family = poisson(), data = dm,
prior = c(
prior(normal(0, 1), class = "Intercept")),
sample_prior = "yes", refresh = 0,
chains = 4, cores = 4)
fit.10h.4.2 <- update(fit.10h.4.1, newdata = dm, refresh = 0,
num_salaman ~ 1 + pct_cover_cs,
prior = c(
prior(normal(0, 1), class = "Intercept"),
prior(normal(0, .5), class = "b")))
fit.10h.4.3 <- update(fit.10h.4.2, newdata = dm, refresh = 0,
num_salaman ~ 1 + pct_cover_cs + age_trees_cs)
fit.10h.4.4 <- update(fit.10h.4.3, newdata = dm, refresh = 0,
num_salaman ~ 1 + pct_cover_cs + log_age_trees_cs)
loo(fit.10h.4.1, fit.10h.4.2, fit.10h.4.3, fit.10h.4.4,
reloo = TRUE) LOOIC SE
fit.10h.4.1 275.75 34.64
fit.10h.4.2 212.76 26.47
fit.10h.4.3 217.02 27.17
fit.10h.4.4 216.74 27.28
fit.10h.4.1 - fit.10h.4.2 62.98 19.24
fit.10h.4.1 - fit.10h.4.3 58.73 18.83
fit.10h.4.1 - fit.10h.4.4 59.01 18.81
fit.10h.4.2 - fit.10h.4.3 -4.25 1.39
fit.10h.4.2 - fit.10h.4.4 -3.97 1.05
fit.10h.4.3 - fit.10h.4.4 0.28 1.08We can interpret the winning model like so:
- When the percent of coverage is at its average value (60.3), we’ll find 1.7 salamanders per plot.
- A single standard deviation change in coverage (+/- 35.1) will cause a 2.76 increase in the rate of salamanders found per plot.n
fixef(fit.10h.4.2)[,1] %>%
exp Intercept pct_cover_cs
1.709934 2.753260 posterior_samples(fit.10h.4.2) %>%
select(starts_with("b_"), starts_with("prior")) %>%
gather(parameter, value) %>%
mutate(value = exp(value)) %>%
ggplot(aes(y=parameter, x = value)) +
geom_halfeyeh() +
theme_bw() +
ggtitle("Posterior distribution for effect estimates")
Frequentist approach
Just to gut check my bayesian findings, lets prove that the simpler model works
library(rsample)
library(yardstick)
formula.int <- formula(num_salaman ~ 1)
formula.cover.cs <- formula(num_salaman ~ 1 + pct_cover_cs)
formula.cover.trees <- formula(num_salaman ~ 1 + pct_cover_cs + age_trees)
formula.cover.logtrees <- formula(num_salaman ~ 1 + pct_cover_cs + log_age_trees)
formula.covercs.logtreescs <- formula(num_salaman ~ 1 + pct_cover_cs + log_age_trees_cs)
fit_model <- function(splits, formula) {
fit.none <- glm(formula,
family = poisson(), data = analysis(splits))
holdout <- assessment(splits)
preds <- predict(fit.none, newdata = holdout)
rmse_vec(holdout$num_salaman, preds)
}
bootstraps(dm, times = 1000) %>%
mutate(
intercept = map_dbl(splits, ~ fit_model(.x, formula.int)),
cover_cs = map_dbl(splits, ~ fit_model(.x, formula.cover.cs)),
cover_trees = map_dbl(splits, ~ fit_model(.x, formula.cover.trees)),
cover_logtrees = map_dbl(splits, ~ fit_model(.x, formula.cover.logtrees)),
covercs_logtreescs = map_dbl(splits, ~ fit_model(.x, formula.covercs.logtreescs))) %>%
as.data.frame %>%
select(-splits, -id) %>%
gather(model, rmse) %>%
group_by(model) %>%
mean_hdi() %>%
arrange(rmse)LS0tCnRpdGxlOiAiQ2hhcHRlciAxMSAtIEdvZCBzcGlrZWQgdGhlIGludGVnZXJzIgpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sKLS0tCgpgYGB7ciBzZXR1cH0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KAoJZmlnLmhlaWdodCA9IDIuNSwKCWZpZy53aWR0aCA9IDQuMiwKCW1lc3NhZ2UgPSBGQUxTRSwKCXdhcm5pbmcgPSBGQUxTRQopCiAKbGlicmFyeSh0aWR5dmVyc2UpCmxpYnJhcnkobWFncml0dHIpCmxpYnJhcnkoYnJtcykKbGlicmFyeSh0aWR5YmF5ZXMpCmxpYnJhcnkoYmF5ZXNwbG90KQoKZGV2dG9vbHM6OmxvYWRfYWxsKCkKCmxpYnJhcnkocmNhcnRvY29sb3IpCmNvbG91cl90aGVtZSA8LSAiQnVyZ1lsIgpwYWxldHRlIDwtIGNhcnRvX3BhbCg3LCBjb2xvdXJfdGhlbWUpCmBgYAoKIyBCaW5vbWlhbCBSZWdyZXNzaW9uCgpUaGUgYmlub21pYWwgZGlzdHJpYnV0aW9uIGlzIGRlbm90ZWQgJHkgXHNpbSBCaW5vbWlhbChuLCBwKSQuICR5JCBpcyBhIGNvdW50LCAkcCQgaXMgdGhlIHByb2JhYmlsaXR5IGFueSBwYXJ0aWN1bGFyICJ0cmlhbCIgaXMgYSBzdWNjZXNzLCBhbmQgJG4kIGlzIHRoZSBudW1iZXIgb2YgdHJpYWxzLiBUaGUgY29tZSBpbiB0d28gZmxhdm9ycyB0aGF0IGJvdGggdXNlIHRoZSBsb2dpdCBsaW5rIGZ1bmN0aW9uLgoKMS4gX0xvZ2lzdGljIFJlZ3Jlc3Npb25fIGlzIHVzZWQgd2hlbiBkYXRhIGlzIG9yZ2FuaXplZCBpbiB0byBzaW5nbGUtdHJpYWwgY2FzZXMgYW5kIHRoZSBvdXRjb21lIHZhcmlhYmxlIGlzIGVpdGhlciAwIG9yIDEuCjIuIF9BZ2dyZWdhdGVkIGJpbm9taWFsIHJlZ3Jlc3Npb25fIGlzIHVzZWQgd2hlbiBpbmRpdmlkdWFsIHRyaWFscyB3aXRoIHRoZSBzYW1lIGNvdmFyaWF0ZXMgYXJlIGFnZ3JlZ2F0ZWQgdG9nZXRoZXIuIEluIHRoaXMgY2FzZSB0aGUgYWdncmVnYXRlZCBvdXRjb21lIGlzIGFuIGludGVnZXIgY291bnQgb2Ygc3VjY2Vzc2VzLgoKIyMgTG9naXN0aWMgcmVncmVzc2lvbiBleGFtcGxlLgoKTGV0cyBwcmVwYXJlIGRhdGEgZnJvbSBhIGNoaW1wYW56ZWUgc29jaWFsIGV4cGVyaW1lbnQuIFdlIHVzZSB0aGUgYHJlY2lwZXNgIHBhY2thZ2UgdG8gZXhwbGljaXRseSBjcmVhdGUgbGVzcyB0aGFuIGZ1bGwgcmFuayBkdW1teSB2YXJpYWJsZXMuCmBgYHtyfQpsaWJyYXJ5KHJlY2lwZXMpCmxpYnJhcnkocmV0aGlua2luZykKZGF0YShjaGltcGFuemVlcykKZCA8LSBhc19kYXRhX2ZyYW1lKGNoaW1wYW56ZWVzKSAlPiUgCiAgc2VsZWN0KHB1bGxlZF9sZWZ0LCBwcm9zb2NfbGVmdCwgY29uZGl0aW9uLCBhY3RvcikKcm0oY2hpbXBhbnplZXMpCmRldGFjaCgicGFja2FnZTpyZXRoaW5raW5nIikKCmQgPC0gZCAlPiUgCiAgbXV0YXRlKAogICAgdHJlYXRtZW50ID0gMSArIHByb3NvY19sZWZ0ICsgMiAqIGNvbmRpdGlvbiwKICAgIHRyZWF0bWVudCA9IGNhc2Vfd2hlbigKICAgICAgdHJlYXRtZW50ID09IDEgfiAiUi9OIiwKICAgICAgdHJlYXRtZW50ID09IDIgfiAiTC9OIiwKICAgICAgdHJlYXRtZW50ID09IDMgfiAiUi9QIiwKICAgICAgdHJlYXRtZW50ID09IDQgfiAiTC9QIiksCiAgICBhY3RvciA9IGFzLmZhY3RvcihhY3RvcikpCgpkZiA8LSByZWNpcGUocHVsbGVkX2xlZnQgfiAwICsgdHJlYXRtZW50ICsgYWN0b3IsIGRhdGEgPSBkKSAlPiUgCiAgc3RlcF9kdW1teSh0cmVhdG1lbnQsIGFjdG9yLCBvbmVfaG90ID0gVFJVRSkgJT4lIAogIHByZXAoKSAlPiUgCiAgYmFrZShuZXdfZGF0YSA9IGQpCgpkZiAlPiUgCiAgc2tpbXI6OnNraW0oKQpgYGAKClJlbWVtYmVyLCBvdXIgY29lZmZpY2llbnRzIHVsdGltYXRlbHkgbmVlZCB0byBiZSBpbnRlcnByZXRlZCBpbiB0aGUgc3BhY2Ugb24gdGhlIG90aGVyIHNpZGUgb2YgdGhlIGxpbmsgZnVuY3Rpb24uIFdoYXQgd291bGQgYSBmbGF0IHByaW9yIHRoYXQgd2UgdXNlZCBpbiB0aGUgZ2F1c3NpYW4gd29ybGQsIGxpa2UgJG5vcm1hbCgwLCAxMCkkLCBtZWFuIGluIHRoZSBsb2dpc3RpYyB3b3JsZD8gV2VsbCB0b28gbXVjaCBvZiB0aGF0IGRlbnNpdHkgd291bGQgYmUgcGlsZWQgdXAgb3V0c2lkZSB0aGUgYm91bmRzIG9mIHByb2JhYmlsaXR5IG1ha2luZyBhbnl0aGluZyBidXQgMSBhbmQgMCBwcm9iYWJsZS4gKkEgZmxhdCBwcmlvciBpbiB0aGUgbG9naXQgc3BhY2UgaXMgbm90IGEgZmxhdCBwcmlvciBpbiB0aGUgb3V0Y29tZSBwcm9iYWJpbGl0eSBzcGFjZS4qCgpgYGB7cn0Kcm5vcm0oMWU0LCAwLCAxMCkgJT4lIAogIGludl9sb2dpdF9zY2FsZWQoKSAlPiUKICB0aWJibGU6OmVuZnJhbWUoKSAlPiUgCiAgZ2dwbG90KGFlcyh2YWx1ZSkpICsKICAgIGdlb21fZGVuc2l0eShmaWxsID0gcGFsZXR0ZVs3XSwgY29sb3VyID0gInRyYW5zcGFyZW50IikgKwogICAgdGhlbWVfYnVyZ3lsKCkgKwogICAgZ2d0aXRsZSgiSW1wcm9wZXIgcHJpb3IgLSBub3JtYWwoMCwgMTApIiwgCiAgICAgICAgICAgICJQcmlvcnMgbmVlZCB0byBiZSBjYWxsaWJyYXRlZCB0byB0aGUgb3V0Y29tZSBwcm9iYWJpbGl0eSBzcGFjZSIpCmBgYAoKTGV0cyB0cnkgc29tZXRoaW5nIG1vcmUgcmVhc29uYWJsZS4KYGBge3J9CnJub3JtKDFlNCwgMCwgMS41KSAlPiUgCiAgaW52X2xvZ2l0X3NjYWxlZCgpICU+JSAKICB0aWJibGU6OmVuZnJhbWUoKSAlPiUgCiAgZ2dwbG90KGFlcyh2YWx1ZSkpICsKICAgIGdlb21fZGVuc2l0eShmaWxsID0gcGFsZXR0ZVs3XSwgY29sb3VyID0gInRyYW5zcGFyZW50IikgKwogICAgdGhlbWVfYnVyZ3lsKCkgKwogICAgZ2d0aXRsZSgiQSBtb3JlIHByb3BlciBwcmlvciIsICJub3JtYWwoMCwgMS41KSIpCmBgYAoKTm93IHdlIGZpdCB0aGUgZGF0YSB3aXRoIGEgcGFyYW1ldGVyIGZvciBlYWNoIGFjdG9yIGFuZCBlYWNoIHBvc3NpYmxlIHRyZWF0bWVudC4KYGBge3J9CmIxMC4xIDwtCiAgYnJtKGRhdGEgPSBkZiwgZmFtaWx5ID0gYmVybm91bGxpLAogICAgICBwdWxsZWRfbGVmdCB+IDAgKyAuLAogICAgICBwcmlvciA9IHByaW9yKG5vcm1hbCgwLCAxLjUpLCBjbGFzcyA9IGIpLAogICAgICByZWZyZXNoID0gMCwgY29yZXMgPSA0KQpzdW1tYXJ5KGIxMC4xKQpgYGAKCkJlbG93IHdlIGV4YW1pbmUgb3VyIGNvZWZmaWNpZW50cy4gV2UgbmVlZCB0byBjb252ZXJ0IGVhY2ggcGFyYW1ldGVyIHRvIHRoZSBvdXRjb21lIHNjYWxlLiBXZSBjYW4gZG8gdGhpcyB3aXRoIGBicm1zOjppbnZfbG9naXRfc2NhbGVkYC4gTm90ZSBob3cgYW50aXNvY2lhbCBhY3RvciAyIGlzLgpgYGB7cn0KcG9zdCA8LSBwb3N0ZXJpb3Jfc2FtcGxlcyhiMTAuMSkgJT4lIAogIGdhdGhlcihwYXJhbWV0ZXIsIHZhbHVlKSAlPiUgCiAgbXV0YXRlKHZhbHVlID0gaW52X2xvZ2l0X3NjYWxlZCh2YWx1ZSkpCgpwX2FjdG9ycyA8LSBwb3N0ICU+JSAKICBmaWx0ZXIoc3RyX2RldGVjdChwYXJhbWV0ZXIsICJiX2FjdG9yLioiKSkgJT4lIAogIGdncGxvdChhZXMoeT1wYXJhbWV0ZXIsIHg9dmFsdWUpKSArCiAgICB0aWR5YmF5ZXM6Omdlb21faGFsZmV5ZWgoZmlsbCA9IHBhbGV0dGVbM10pICsKICAgIHRoZW1lX2J1cmd5bCgpCgpwX3RyZWF0IDwtIHBvc3QgJT4lIAogIGZpbHRlcihzdHJfZGV0ZWN0KHBhcmFtZXRlciwgImJfdHJlYXQuKiIpKSAlPiUgCiAgZ2dwbG90KGFlcyh5PXBhcmFtZXRlciwgeCA9IHZhbHVlKSkgKwogICAgZ2VvbV9oYWxmZXllaChmaWxsID0gcGFsZXR0ZVs1XSkgKwogICAgdGhlbWVfYnVyZ3lsKCkKCmdyaWRFeHRyYTo6Z3JpZC5hcnJhbmdlKHBfYWN0b3JzLCBwX3RyZWF0LCBuY29sID0gMikKYGBgCgpXZSBjYW4gbm93IGNvbXBhcmUgdGhlIGNvbnN0cmFzdHMgaW4gdHJlYXRtZW50cy4KYGBge3J9CnBvc3Rlcmlvcl9zYW1wbGVzKGIxMC4xKSAlPiUgCiAgc2VsZWN0KHN0YXJ0c193aXRoKCJiX3RyZWF0IikpICU+JSAKICB0cmFuc211dGUoCiAgICBgIGxlZnQgbGV2ZXJgID0gKGJfdHJlYXRtZW50X0wuTiAtIGJfdHJlYXRtZW50X0wuUCksCiAgICBgIHJpZ2h0IGxldmVyYCA9IChiX3RyZWF0bWVudF9SLk4gLSBiX3RyZWF0bWVudF9SLlApKSAlPiUgCiAgZ2F0aGVyKHBhcmFtZXRlciwgdmFsdWUpICU+JQogIGdncGxvdChhZXMoeSA9IHBhcmFtZXRlciwgZmlsbCA9IHBhcmFtZXRlciwgeCA9IHZhbHVlKSkgKwogICAgZ2VvbV9oYWxmZXllaCgud2lkdGggPSBjKC41LCAuOTUpKSArCiAgICBnZW9tX3ZsaW5lKHhpbnRlcmNlcHQgPSAwLCBjb2xvdXIgPSBhbHBoYShwYWxldHRlWzddLCAuNiksIGxpbmV0eXBlID0gMikgKwogICAgdGhlbWVfYnVyZ3lsKCkgKwogICAgc2NhbGVfZmlsbF9tYW51YWwodmFsdWVzID0gYyhwYWxldHRlWzNdLCBwYWxldHRlWzVdKSkgKwogICAgc2NhbGVfeF9jb250aW51b3VzKGJyZWFrcyA9IHJvdW5kKHNlcShmcm9tPS0uNiwgdG8gPSAxLjQsIGJ5ID0gLjIpLDEpKSArCiAgICB0aGVtZShsZWdlbmQudGl0bGUgPSBlbGVtZW50X2JsYW5rKCksCiAgICAgICAgICBheGlzLnRleHQueSA9IGVsZW1lbnRfYmxhbmsoKSwKICAgICAgICAgIGF4aXMudGlja3MueSA9IGVsZW1lbnRfYmxhbmsoKSkgKwogICAgICAgICAgeWxhYigiIikgKyB4bGFiKCJMb2cgb2RkcyBvZiBwdWxsaW5nIHRoZSBsZWZ0IGxldmVyIikgKwogICAgZ2d0aXRsZSgiUHJvc29jaWFsIGJlaGF2aW9yIGlzIG5vdCBzdGF0aXN0aWNhbGx5IGRpc3Rpbmd1aXNoYWJsZSIpCmBgYAoKIyMjIEFnZ3JlZ2F0ZWQgYmlub21pYWwKCldoZW4gdGhhdCBzYW1lIGRhdGEgaXMgYWdncmVnYXRlZCwgd2UgY2FuIHN3aXRjaCB0byBhIGJpbm9taWFsIHJlZ3Jlc3Npb24gaW5zdGVhZCBvZiBhIGJlcm5vdWxpLiBOb3RlIHRoZSB1c2Ugb2YgdGhlIGBjcml0ZXJpYSB8IHRyaWFscygpYCBzeW50YXguCmBgYHtyfQpsaWJyYXJ5KHJldGhpbmtpbmcpCmRhdGEoY2hpbXBhbnplZXMpCmQgPC0gY2hpbXBhbnplZXMKZGV0YWNoKCJwYWNrYWdlOnJldGhpbmtpbmciKQoKZF9hZ2dyZWdhdGVkIDwtIGQgJT4lCiAgc2VsZWN0KC1yZWNpcGllbnQsIC1ibG9jaywgLXRyaWFsLCAtY2hvc2VfcHJvc29jKSAlPiUKICBncm91cF9ieShhY3RvciwgY29uZGl0aW9uLCBwcm9zb2NfbGVmdCkgJT4lCiAgc3VtbWFyaXNlKHggPSBzdW0ocHVsbGVkX2xlZnQpKQoKZF9hZ2dyZWdhdGVkICU+JQogIGZpbHRlcihhY3RvciAlaW4lIGMoMSwgMikpCmBgYAoKYGxvb2AgaXMgbWlzbGVhZGluZyBpbiB0aGUgYWdncmVnYXRlIGNhc2UsIHNwZWNpZmljYWxseSBiZWNhdXNlIHlvdSBhcmUgbGVhdmluZyBvdXQgYWdncmVnYXRpb25zIGF0IGEgdGltZS4gU3RpY2sgdG8gdGhlIGJlcm5vdWxsaSBpZiB5b3Ugd2FudCB0byB1c2UgYGxvb2AuCmBgYHtyfQpiMTAuNSA8LQogIGJybShkYXRhID0gZF9hZ2dyZWdhdGVkLCBmYW1pbHkgPSBiaW5vbWlhbCwKICAgICAgeCB8IHRyaWFscygxOCkgfiAxICsgcHJvc29jX2xlZnQgKyBjb25kaXRpb246cHJvc29jX2xlZnQgLAogICAgICBwcmlvciA9IGMocHJpb3Iobm9ybWFsKDAsIDEwKSwgY2xhc3MgPSBJbnRlcmNlcHQpLAogICAgICAgICAgICAgICAgcHJpb3Iobm9ybWFsKDAsIDEwKSwgY2xhc3MgPSBiKSksCiAgICAgIGl0ZXIgPSAyNTAwLCB3YXJtdXAgPSA1MDAsIGNvcmVzID0gMiwgY2hhaW5zID0gMiwKICAgICAgcmVmcmVzaCA9IDApCmxvbyhiMTAuNSkKYGBgCgojIyMgR3JhZHVhdGUgc2Nob29sIGFkbWlzc2lvbnMKCkVhY2ggcm93IGlzIGEgYGRlcGFydG1lbnQtZ2VuZGVyYCB3aXRoIHRoZSBudW1iZXIgb2YgYWNjZXB0YW5jZXMgb3V0IG9mIHRvdGFsIGFwcGxpY2F0aW9ucy4gVGhpcyBpcyBqdXN0IGFuIGFnZ3JlZ2F0ZWQgYmVybm91bGxpIHRoYXQgd2Ugd2lsbCByZXByZXNlbnQgYXMgaW5kZXBlbmRlbnQgYmlub21pYWwgdHJpYWxzIHRvIGNvbGxlY3QgdGhlIGVmZmVjdCBvZiBnZW5kZXIuCmBgYHtyfQpsaWJyYXJ5KHJldGhpbmtpbmcpCmRhdGEoVUNCYWRtaXQpCmQgPC0gVUNCYWRtaXQKZGV0YWNoKHBhY2thZ2U6cmV0aGlua2luZykKbGlicmFyeShicm1zKQpybShVQ0JhZG1pdCkKZCA8LSAKICBkICU+JQogIG11dGF0ZSgKICAgIG1hbGUgPSBpZmVsc2UoYXBwbGljYW50LmdlbmRlciA9PSAibWFsZSIsIDEsIDApLAogICAgZmVtYWxlID0gLTEqbWFsZSArIDEpCmQgJT4lIAogIGdsaW1wc2UKYGBgCgpIZXJlIGlzIHRoZSBtb2RlbCB3ZSBhcmUgcnVubmluZy4gV2UgdXNlIGEgcmVsYXRpdmVseSBmbGF0IHByaW9yIG9uIHRoZSBvdXRjb21lIHNjYWxlLgokJAphZG1pc3Npb24gXHNpbSBCaW5vbWlhbChOX2ksIHBfaSkKXFwKbG9naXQocF9pKSA9IFxhbHBoYSAqIG1hbGUgKyBcYmV0YSAqIGZlbWFsZQpcXApcYWxwaGEsIFxiZXRhIFxzaW0gTm9ybWFsKDAsIDEuNSkKJCQKCmBgYHtyfQpiMTAuNiA8LQogIGJybShkYXRhID0gZCwgZmFtaWx5ID0gYmlub21pYWwsCiAgICAgIGFkbWl0IHwgdHJpYWxzKGFwcGxpY2F0aW9ucykgfiAwICsgbWFsZSArIGZlbWFsZSwKICAgICAgcHJpb3IgPSBjKHByaW9yKG5vcm1hbCgwLCAuMTUpLCBjbGFzcyA9IGIpKSwKICAgICAgaXRlciA9IDI1MDAsIHdhcm11cCA9IDUwMCwgY29yZXMgPSA0LCBjaGFpbnMgPSA0LAogICAgICByZWZyZXNoID0gMCwgc2FtcGxlX3ByaW9yID0gInllcyIpCnBsb3QoYjEwLjYpCmBgYAoKYGBge3J9CnByaW9yX3NhbXBsZXMoYjEwLjYpICU+JSAKICBnYXRoZXIocGFyYW1ldGVyKSAlPiUgCiAgZ2dwbG90KGFlcyh5ID0gcGFyYW1ldGVyLCB4ID0gaW52X2xvZ2l0X3NjYWxlZCh2YWx1ZSkpKSArCiAgICBnZW9tX2hhbGZleWVoKC53aWR0aCA9IGMoLjk1LCAuNSksCiAgICAgICAgICAgICAgICAgIGZpbGwgPSBwYWxldHRlWzVdLAogICAgICAgICAgICAgICAgICBwb2ludF9pbnRlcnZhbCA9IG1lZGlhbl9oZGkpICsKICAgIHRoZW1lX2J1cmd5bCgpICsKICAgIGdndGl0bGUoIldlYWtseSByZWd1bGFyaXppbmcgcHJpb3IgZm9yIGNvZWZmaWNpZW50cyIsCiAgICAgICAgICAgICI5NSUgYW5kIDUwJSBoaWdoZXN0IGRlbnNpdHkgaW50ZXJ2YWwgYWJvdXQgdGhlIG1lZGlhbiIpICAKYGBgCgpgYGB7cn0KcG9zdGVyaW9yX3NhbXBsZXMoYjEwLjYpICU+JSAKICBzZWxlY3Qoc3RhcnRzX3dpdGgoImJfIikpICU+JQogIGdhdGhlcihwYXJhbWV0ZXIpICU+JSAKICBnZ3Bsb3QoYWVzKHZhbHVlLCBwYXJhbWV0ZXIsIGZpbGwgPSBwYXJhbWV0ZXIpKSArCiAgICBnZW9tX2hhbGZleWVoKCkgKwogICAgdGhlbWVfYnVyZ3lsKCJ0bCIpICsgeWxhYigiIikgKwogICAgc2NhbGVfZmlsbF9tYW51YWwodmFsdWVzID0gYyhwYWxldHRlWzNdLCBwYWxldHRlWzZdKSkgKwogICAgdGhlbWUobGVnZW5kLnRpdGxlID0gZWxlbWVudF9ibGFuaygpLAogICAgICAgICAgYXhpcy50aWNrcy55ID0gZWxlbWVudF9ibGFuaygpLAogICAgICAgICAgYXhpcy50ZXh0LnkgPSBlbGVtZW50X2JsYW5rKCkpCmBgYAoKCk5vdyBsZXRzIGNvbXB1dGUgdGhlIGNvbnRyYXN0cy4gCmBgYHtyfQpwb3N0ZXJpb3Jfc2FtcGxlcyhiMTAuNikgJT4lIAogIHRyYW5zbXV0ZSgKICAgIGBsb2dpdCBzY2FsZWAgPSBiX21hbGUgLSBiX2ZlbWFsZSwKICAgIGBvdXRjb21lIHNjYWxlYCA9IGludl9sb2dpdF9zY2FsZWQoYl9tYWxlKSAtIGludl9sb2dpdF9zY2FsZWQoYl9mZW1hbGUpKSAlPiUgCiAgZ2F0aGVyKHBhcmFtZXRlcikgJT4lIAogIGdncGxvdChhZXMoeSA9IHBhcmFtZXRlciwgeCA9IHZhbHVlLCBmaWxsID0gcGFyYW1ldGVyKSkgKwogICAgZ2VvbV9oYWxmZXllaChwb2ludF9pbnRlcnZhbCA9IG1lZGlhbl9oZGksCiAgICAgICAgICAgICAgICAgIC53aWR0aCA9IGMoLjk1LCAuNSkpICsKICAgIGdndGl0bGUoIk1lbiBoYXZlIGEgMTIlIGhpZ2hlciBhY2NlcHRhbmNlIHJhdGUgdGhhbiB3b21lbiIsCiAgICAgICAgICAgICI5NSUgYW5kIDUwJSBoaWdoZXN0IGRlbnNpdHkgaW50ZXJ2YWwgYWJvdXQgdGhlIG1lZGlhbiIpICsKICAgIHRoZW1lX2J1cmd5bCgidGwiKSArIHlsYWIoIiIpICsgCiAgICBzY2FsZV9maWxsX21hbnVhbCh2YWx1ZXMgPSBjKHBhbGV0dGVbM10sIHBhbGV0dGVbNl0pKSArCiAgICB0aGVtZV9idXJneWwoKSArCiAgICB0aGVtZShheGlzLnRleHQueSA9IGVsZW1lbnRfYmxhbmsoKSwKICAgICAgICAgIGF4aXMudGlja3MueSA9IGVsZW1lbnRfYmxhbmsoKSkKYGBgCgpJIHBlcnNvbmFsbHkgZmluZCB0aGF0IHRoZSBpbnRlcnByZXRhdGlvbiBpcyBtb3JlIGludGVycHJldGFibGUgb24gdGhlIG91dGNvbWUgc2NhbGUuCmBgYHtyfQpzcHJlYWRfZHJhd3MoYjEwLjYsIGJfbWFsZSwgYl9mZW1hbGUpICU+JSAKICB0cmFuc211dGUoZGlmZiA9IGludl9sb2dpdF9zY2FsZWQoYl9tYWxlKSAtIGludl9sb2dpdF9zY2FsZWQoYl9mZW1hbGUpKSAlPiUgCiAgbWVkaWFuX2hkaSgpCmBgYAoKV2hhdCBkaWQgd2UganVzdCBkbyBoZXJlPyBSZWNhbGwgdGhhdCB0aGlzIG91ciBmb3JtdWxhIGZvciBhIGxvZ2lzdGljIHJlZ3Jlc3Npb24uCiQkCmxvZ2l0KHApID0gLS4yMDYyNzMwICogaXNcX21hbGUgKyAtLjc0NjkxNTMgKiBpc1xfZmVtYWxlCiQkCgpGb3IgZWFjaCBnZW5kZXIsIHdlIHByZWRpY3QgdGhlIGZvbGxvd2luZyBwcm9iYWJpbGl0aWVzLiBUaGF0IDEyIGlzIHRoZSBkaWZmZXJlbmNlIGluIHBlcmNlbnRhZ2UgcG9pbnRzISBOb3QgdGhhdCBtYWxlcyBhcmUgMTIlIG1yb2UgbGlrZWx5IHRvIGdldCBhZG1pdHRlZCEKYGBge3J9CnNwcmVhZF9kcmF3cyhiMTAuNiwgYl9tYWxlLCBiX2ZlbWFsZSkgJT4lIAogIHNlbGVjdChzdGFydHNfd2l0aCgiYl8iKSkgJT4lIAogIGdhdGhlcihwYXJhbWV0ZXIsIHZhbHVlKSAlPiUgCiAgbXV0YXRlKHZhbHVlID0gaW52X2xvZ2l0X3NjYWxlZCh2YWx1ZSkpICU+JSAKICBnZ3Bsb3QoYWVzKHkgPSBwYXJhbWV0ZXIsIHggPSB2YWx1ZSkpICsKICAgIHRpZHliYXllczo6c3RhdF9pbnRlcnZhbGgocG9pbnRfaW50ZXJ2YWwgPSBtZWRpYW5faGRpKQpgYGAKCkZpbmFsbHksIGxldHMgcGVyZm9ybSBhIHBvc3RlcmlvciBwcmVkaWN0aW9uIGNoZWNrLiBJdCB0dXJucyBvdXQgd2UgYXJlIHByZXR0eSBvZmYuCgo+IFRoZSBwcm9ibGVtIGluIHRoaXMgY2FzZSBpcyB0aGF0IG1hbGVzIGFuZCBmZW1hbGVzIGRvIG5vdCBhcHBseSB0byB0aGUgc2FtZSBkZXBhcnRtZW50cywgYW5kIGRlcGFydG1lbnRzIHZhcnkgaW4gdGhlaXIgcmF0ZXMgb2YgYWRtaXNzaW9uLiBUaGlzIG1ha2VzIHRoZSBhbnN3ZXIgbWlzbGVhZGluZy4gWW91IGNhbiBzZWUgdGhlIHN0ZWFkeSBkZWNsaW5lIGluIGFkbWlzc2lvbiBwcm9iYWJpbGl0eSBmb3IgYm90aCBtYWxlcyBhbmQgZmVtYWxlcyBmcm9tIGRlcGFydG1lbnQgQSB0byBkZXBhcnRtZW50IEYuIEZlbWFsZXMgaW4gdGhlc2UgZGF0YSB0ZW5kZWQgbm90IHRvIGFwcGx5IHRvIGRlcGFydG1lbnRzIGxpa2UgQSBhbmQgQiwgd2hpY2ggaGFkIGhpZ2ggb3ZlcmFsbCBhZG1pc3Npb24gcmF0ZXMuIEluc3RlYWQgdGhleSBhcHBsaWVkIGluIGxhcmdlIG51bWJlcnMgdG8gZGVwYXJ0bWVudHMgbGlrZSBGLCB3aGljaCBhZG1pdHRlZCBsZXNzIHRoYW4gMTAlIG9mIGFwcGxpY2FudHMuIFNvIHdoaWxlIGl0IGlzIHRydWUgb3ZlcmFsbCB0aGF0IGZlbWFsZXMgaGFkIGEgbG93ZXIgcHJvYmFiaWxpdHkgb2YgYWRtaXNzaW9uIGluIHRoZXNlIGRhdGEsIGl0IGlzIGNsZWFybHkgbm90IHRydWUgd2l0aGluIG1vc3QgZGVwYXJ0bWVudHMuIAoKYGBge3J9CmQgJT4lIAogIG11dGF0ZShwcmVkaWN0aW9uID0gcHJlZGljdChiMTAuNilbLDFdLAogICAgICAgICByZXNpZHVhbHMgPSBhZG1pdCAtIHByZWRpY3Rpb24pICU+JSAKICBncm91cF9ieShkZXB0KSAlPiUgCiAgc3VtbWFyaXNlKHJlc2lkID0gbWVhbihyZXNpZHVhbHMpKSAlPiUgCiAgYXJyYW5nZShkZXNjKGFicyhyZXNpZCkpKQpgYGAKCldoYXQgd2Ugd2lsbCBub3cgZG8gaXMgaW5jbHVkZSB2YXJpYWJsZXMgZm9yIGVhY2ggZGVwYXJ0bWVudC4gVGhhdCB3YXkgdGhlIGdlbmRlciB2YXJpYWJsZXMgd2lsbCBvbmx5IHByb3ZpZGUgbWFyZ2luYWwgaW5mb3JtYXRpb24gb24gdG9wIG9mIHRoZSBiYXNlbGluZSBhY2NlcHRhbmNlIHJhdGVzIG9mIGVhY2ggZGVwYXJ0bWVudC4gCgpgYGB7cn0KbGlicmFyeShyZWNpcGVzKQoKZGYgPC0gZCAlPiUKICBzZWxlY3QobWFsZSwgZmVtYWxlLCBkZXB0LCBhZG1pdCwgYXBwbGljYXRpb25zKSAlPiUgCiAgcmVjaXBlKGFkbWl0IH4gYXBwbGljYXRpb25zICsgbWFsZSArIGZlbWFsZSArIGRlcHQpICU+JSAKICAgIHN0ZXBfZHVtbXkoZGVwdCwgb25lX2hvdCA9IFRSVUUpICU+JSAKICAgIHByZXAoKSAlPiUgCiAgICBiYWtlKG5ld19kYXRhPWQpCgpoIDwtIGJmKCkKCmIxMC44IDwtICAgYnJtKGRhdGEgPSBkZiwgZmFtaWx5ID0gYmlub21pYWwsCiAgICAgICAgICAgICAgIGFkbWl0IHwgdHJpYWxzKGFwcGxpY2F0aW9ucykgfiAwICsgbWFsZSArIGZlbWFsZSArIAogICAgICAgICAgZGVwdF9BICsgZGVwdF9CICsgZGVwdF9DICsgZGVwdF9EICsgZGVwdF9FICsgZGVwdF9GLAogICAgICBwcmlvcihub3JtYWwoMCwgLjE1KSwgY2xhc3MgPSBiKSkKCnN1bW1hcnkoYjEwLjgpCmBgYAoKTGV0cyBjb21wdXRlIHRoZSBjb250cmFzdCBhZ2Fpbi4gVGhlIGRpZmZlcmVuY2UgaW4gYWNjZXB0YW5jZSByYXRlcyBpcyBubyBsb25nZXIgc2lnbmlmaWNhbnQuIFdoYXQgaGFwcGVuZWQ/CmBgYHtyfQpwb3N0ZXJpb3Jfc2FtcGxlcyhiMTAuOCkgJT4lIAogIHRyYW5zbXV0ZShtYWxlID0gaW52X2xvZ2l0X3NjYWxlZChiX21hbGUpLAogICAgICAgICAgICBmZW1hbGUgPSBpbnZfbG9naXRfc2NhbGVkKGJfZmVtYWxlKSwKICAgICAgICAgICAgZGlmZiA9IG1hbGUgLSBmZW1hbGUpICU+JSAKICBzZWxlY3QoZGlmZikgJT4lIGdhdGhlcihwYXJhbWV0ZXIpICU+JSAKICBnZ3Bsb3QoYWVzKHk9cGFyYW1ldGVyLCB4PXZhbHVlKSkgKwogICAgZ2VvbV9leWVoKGZpbGwgPSBwYWxldHRlWzZdLAogICAgICAgICAgICAgIC53aWR0aCA9IGMoLjUsIC45NSkpICsKICAgIGdlb21fdmxpbmUoeGludGVyY2VwdCA9IDAsIGxpbmV0eXBlID0gMikgKwogICAgdGhlbWVfYnVyZ3lsKCkKYGBgCgpEZXBhcnRtZW50IGlzIGEgY29uZm91bmQuIEl0IG9wZW5zIHVwIGEgYmFja2Rvb3IgcGlwZSB0byBhY2NlcHRhbmNlIHJhdGVzLgpgYGB7ciBmaWcuaGVpZ2h0PS43NSwgZmlnLndpZHRoPTEuNX0KbGlicmFyeShkYWdpdHR5KQpkYWcgPC0gZGFnaXR0eSgiZGFnIHsKICAgICAgICBnZW5kZXIgLT4gYWNjZXB0YW5jZQogICAgICAgIGRlcGFydG1lbnQgLT4gYWNjZXB0YW5jZQogICAgICAgIGdlbmRlciAtPiBkZXBhcnRtZW50Cn0iKQpwbG90KGRhZ2l0dHk6OmdyYXBoTGF5b3V0KGRhZykpCmBgYAoKIyMjIE11bHRpbm9taWFsIGFuZCBDYXRlZ29yaWNhbCBNb2RlbHMKCldoZW4gbW9yZSB0aGFuIHR3byB0eXBlcyBvZiB1bm9yZGVyZWQgZXZlbnRzIGFyZSBwb3NzaWJsZSwgYW5kIHRoZSBwcm9iYWJpbGl0eSBvZiBlYWNoIHR5cGUgb2YgZXZlbnQgaXMgY29uc3RhbnQgYWNyb3NzIHRyaWFscywgdGhlbiB0aGUgbWF4aW11bSBlbnRyb3B5IGRpc3RyaWJ1dGlvbiBpcyB0aGUgX211bHRpbm9taWFsIGRpc3RyaWJ1dGlvbl8uIE1vZGVsaW5nIHRoaXMgdHlwZSBvZiBkaXN0cmlidXRpb24gY2FuIGJlIGRvbmUgYnkgZWl0aGVyIHVzaW5nIGEgZ2VuZXJhbGl6YXRpb24gb2YgdGhlIGxvZ2l0IGxpbmsgb3IgYnkgdHJhbnNmb3JtaW5nIHRoZSBtdWx0aW5vbWlhbCBsaWtlbGlob29kIGludG8gYSBzZXJpZXMgb2YgUG9pc3NvbiBsaWtlbGlob29kcy4gCgpUaGUgbXVsdGlub21pYWwgbG9naXQgbGluayBmdW5jdGlvbiB0YWtlcyBhIHZlY3RvciBvZiBzY29yZXMsIG9uZSBmb3IgZWFjaCBldmVudCB0eXBlLCBhbmQgY29tcHV0ZXMgdGhlIHByb2JhYmlsaXR5IG9mIGEgc3BlY2lmaWMgZXZlbnQsIGBrYC4KCiQkClByKGt8c18xLCBzXzIsIFxkb3RzLCBzX0spID0gXGZyYWMge2V4cChzX2spfSB7XHN1bV5LX3tpPTF9ZXhwKHNfaSl9CiQkCgpJbiBhIG11bHRpbm9taWFsIEdMTSwgeW91IG5lZWQgdG8gYnVpbGQgYEstMWAgbGluZWFyIG1vZGVscyBmb3IgYEtgIGV2ZW50cywgZWFjaCBvZiB3aGljaCBjYW4gKDEpIHVzZSBhbnkgcHJlZGljdG9yIHBhcmFtZXRlcnMgaXQgd291bGQgbGlrZSBhbmQgKDIpIGdldCBhc3NpZ25lZCBhbnkgcHJlZGljdG9yIHZhbHVlcyBpdCB3YW50cy4KCkV4YW1wbGU6IFByZWRpY3RpbmcgY2FyZWVyIGNob2ljZSBhcyBhIGZ1bmN0aW9uIG9mIGluY29tZS4gSGVyZSAkXGJldGFfe0lOQ09NRX0kIGFwcGVhcnMgaW4gZWFjaCBsaW5lYXIgbW9kZWwgZm9yIGVhY2ggcG9zc2libGUgY2FyZWVyIGNob2ljZS4KYGBge3J9Ck4gICAgICA8LSA1MDAKaW5jb21lIDwtIDE6MwpzY29yZSAgPC0gLjUgKiBpbmNvbWUKCiMgU29mdG1heCBpcyB0aGUgbXVsdGlub21pYWwgbG9naXQgbGluay4gCiMgSXQganVzdCBub3JtYWxpemVzIHZhbHVlcyB0byBiZSBpbiAKIyBwcm9iYWJpbGl0eSBzY2FsZQpwIDwtIHNvZnRtYXgoc2NvcmVbMV0sIHNjb3JlWzJdLCBzY29yZVszXSkKCmNhcmVlciA8LSBtYXBfZGJsKDE6TiwgfiBzYW1wbGUoMTozLCBzaXplPTEsIHByb2IgPSBwKSkKCiMgV2UgY2hvb3NlIHRoZSBmaXJzdCBldmVudCB0byBiZSB0aGUgInJlZmVyZW5jZSIgZXZlbnQuCiMgVGhlIHJlZmVyZW5jZSB0eXBlIGdldHMgYSBjb25zdGFudCwgCiMgYW5kIHRoZSBvdGhlciBldmVudHMgYXJlIGRlZmluZWQgcmVsYXRpdmUgdG8gaXQuCm0xMC4xNiA8LSBtYXAoCiAgICBhbGlzdCgKICAgICAgICAjIG11bHRpbm9taWFsIGxvZ2lzdGljIHJlZ3Jlc3Npb24gZGlzdHJpYnV0aW9uCiAgICAgICAgY2FyZWVyIH4gZGNhdGVnb3JpY2FsKCBzb2Z0bWF4KDAsczIsczMpICksCiAgICAgICAgczIgPC0gYioyLCAgICAjIGxpbmVhciBtb2RlbCBmb3IgZXZlbnQgdHlwZSAyCiAgICAgICAgczMgPC0gYiozLCAgICAjIGxpbmVhciBtb2RlbCBmb3IgZXZlbnQgdHlwZSAzCiAgICAgICAgYiB+IGRub3JtKDAsNSkKKSwgZGF0YT1saXN0KGNhcmVlcj1jYXJlZXIpICkKCnN1bW1hcnkobTEwLjE2KQpgYGAKCmBgYHtyfQpOIDwtIDEwMAojIHNpbXVsYXRlIGZhbWlseSBpbmNvbWVzIGZvciBlYWNoIGluZGl2aWR1YWwKZmFtaWx5X2luY29tZSA8LSBydW5pZihOKQojIGFzc2lnbiBhIHVuaXF1ZSBjb2VmZmljaWVudCBmb3IgZWFjaCB0eXBlIG9mIGV2ZW50CmIgPC0gKDE6LTEpCmNhcmVlciA8LSByZXAoTkEsIE4pICAjIGVtcHR5IHZlY3RvciBvZiBjaG9pY2VzIGZvciBlYWNoIGluZGl2aWR1YWwKZm9yICggaSBpbiAxOk4gKSB7CiAgICBzY29yZSA8LSAwLjUqKDE6MykgKyBiKmZhbWlseV9pbmNvbWVbaV0KICAgIHAgPC0gc29mdG1heChzY29yZVsxXSxzY29yZVsyXSxzY29yZVszXSkKICAgIGNhcmVlcltpXSA8LSBzYW1wbGUoIDE6MyAsIHNpemU9MSAsIHByb2I9cCApCn0KCmRhdGFfZnJhbWUoCiAgY2FyZWVyLAogIGZhbWlseV9pbmNvbWUKKQpgYGAKCkFsdGVybmF0aXZlbHksIHlvdSBjYW4gc3BlY2lmeSBicmFuZCBuZXcgcGFyYW1ldGVycyBmb3IgeW91ciBsaW5lYXIgbW9kZWxzLiBJbnRlcnByZXRhdGlvbiBpcyB2ZXJ5IGRpZmZpY3VsdCBmb3IgbW9kZWxzIGxpa2UgdGhlc2UuCmBgYHtyfQptMTAuMTcgPC0gbWFwKAogICAgYWxpc3QoCiAgICAgICAgY2FyZWVyIH4gZGNhdGVnb3JpY2FsKHNvZnRtYXgoMCxzMixzMykpLAogICAgICAgIHMyIDwtIGEyICsgYjIqZmFtaWx5X2luY29tZSwKICAgICAgICBzMyA8LSBhMyArIGIzKmZhbWlseV9pbmNvbWUsCiAgICAgICAgYyhhMiwgYTMsIGIyLCBiMykgfiBkbm9ybSgwLDUpCiksIGRhdGE9bGlzdChjYXJlZXI9Y2FyZWVyLGZhbWlseV9pbmNvbWU9ZmFtaWx5X2luY29tZSkpCgptMTAuMTcKYGBgCgojIFBvaXNzb24gcmVncmVzc2lvbgoKT2Z0ZW4gdGltZXMsIHRoZSB1cHBlciBsaW1pdCBvZiB5b3VyIGJpbm9taWFsLCAkbiQsIGlzIHVua25vd24uIEluIHRoZXNlIGNhc2VzIHlvdSdkIHdhbnQgdG8gdXNlIGEgcG9pc3NvbiBkaXN0cml1dGlvbi4gUmVjYWxsIHRoYXQgYSBiaW5vbWlhbCBkaXN0cmlidXRpb25zIG1lYW4gaXMgJE5wJCBhbmQgaXRzIHZhcmlhbmNlIGlzICROcCgxLXApJC4gV2hlbiBOIGlzIHZlcnkgbGFyZ2UgYW5kIHAgaXMgdmVyeSBzbWFsbCB0aGVuIHRoZXNlIHR3byBhcmUgYXBwcm94aW1hdGVseSB0aGUgc2FtZS4KYGBge3J9CnkgPC0gcmJpbm9tKDFlNSwgMTAwMCwgMS8xMDAwKQp0aWJibGUoCiAgbWVhbiA9IHJvdW5kKG1lYW4oeSksIDIpLAogIHZhciA9IHJvdW5kKHZhcih5KSwgMikpCmBgYApUaGlzIHVuaWZpZWQgcGFyYW1ldGVyIGlzIHJlcHJlc2VudGVkIGJ5IHRoZSBwb2lzc29uIGRpc3RyaWJ1dGlvbidzIHNvbGUgcGFyYW1ldGVyICRcbGFtYmRhJCB3aGljaCBpcyB0aGUgZXhwZWN0ZWQgdmFsdWUgb2YgdGhlIG91dGNvbWUgeS4gSXQgaXMgYWxzbyB0aGUgZXhwZWN0ZWQgdmFyaWFuY2Ugb2YgdGhlIGNvdW50cyBvZiB5LiBUaGUgY29udmVudGlvbmFsIGxpbmsgZnVuY3Rpb24gZm9yIGEgUG9pc3NvbiBtb2RlbCBpcyB0aGUgbG9nIGxpbmssIHdoaWNoIGVuc3VyZXMgb3VyIGNvdW50IGlzIGFsd2F5cyBwb3NpdGl2ZS4KJCQKeV9pIFxzaW0gUG9pc3NvbihcbGFtYmRhX2kpClxcCmxvZyhcbGFtYmRhX2kpICsgXGFscGhhICsgXGJldGEoeF9pIC0gXGJhciB4KQokJApUaGUgbG9nIGxpbmsgYWxzbyBpbXBsaWVzIGFuIGV4cG9uZW50aWFsIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHByZWRpY3RvcnMgYW5kIHRoZSBleHBlY3RlZCB2YWx1ZS4gCgojIyMgRXhhbXBsZTogT2NlYW5pYyB0b29sIGNvbXBsZXhpdHkKCkhlcmUgd2UgZXhhbWluZSBob3cgcG9wdWxhdGlvbiBzaXplIGRldGVybWluZXMgY29tcGxleGl0eSBvZiB0b29scyBkZXZlbG9wZWQuCmBgYHtyfQpzdXBwcmVzc1BhY2thZ2VTdGFydHVwTWVzc2FnZXMobGlicmFyeShyZXRoaW5raW5nKSkKZGF0YShLbGluZSkKZCA8LSBhc19kYXRhX2ZyYW1lKEtsaW5lKQpybShLbGluZSkKZGV0YWNoKCJwYWNrYWdlOnJldGhpbmtpbmciKQpkZiA8LSBkICU+JSAKICBtdXRhdGUoCiAgICBsb2dfcG9wID0gc2NhbGUobG9nKHBvcHVsYXRpb24pLCBjZW50ZXIgPSBUUlVFKSwKICAgIGNvbnRhY3RfaGlnaCA9IGlmZWxzZShjb250YWN0ID09ICJoaWdoIiwgMSwgMCksCiAgICBjb250YWN0X2xvdyA9IGlmZWxzZShjb250YWN0ID09ICJsb3ciLCAxLCAwKSkKZGYgJT4lIAogIHNlbGVjdChjdWx0dXJlLCBwb3B1bGF0aW9uLCBsb2dfcG9wLAogICAgICAgICBjb250YWN0X2hpZ2gsIGNvbnRhY3RfbG93LCB0b3RhbF90b29scykgJT4lIAogIHNraW1yOjpza2ltKCkKYGBgCgpIZXJlIGlzIHRoZSBtb2RlbCB3ZSdsbCBmaXQ6CiQkCnRvdGFsXF90b29scyBcc2ltIFBvaXNzb24oXGxhbWJkYV9pKQpcXApsb2coXGxhbWJkYV9pKSA9IFxhbHBoYV97Y29udGFjdHtbaV19fSArIFxiZXRhX3tjb250YWN0W2ldfSpsb2cocG9wdWxhdGlvbl9pKQpcXApcYWxwaGFfaiBcc2ltIG5vcm1hbCgzLCAuNSkKXFwKXGJldGFfaiBcc2ltIG5vcm1hbCgwLCAuMikKJCQKCldlIGNob3NlIG91ciBwcmlvcnMgdG8gbWFrZSBzdXJlIHRoYXQgdGhleSBhcmUgc2Vuc2libGUgaW4gdGhlIG91dGNvbWUgc3BhY2UuIFRoZSBwdXJwb3NlIG9mIG91ciBtb3JlIHNwaWtlZCBiZXRhIHByaW9yIGlzIHRvIGRpc3N1YWRlIG91ciBtb2RlbCBmcm9tIGJlbGlldmluZyBleHBsb3NpdmUgZXhwb25lbnRpYWwgcmVsYXRpb25zaGlwcy4KYGBge3J9CnRpYmJsZSgKICBhbHBoYSA9IHJub3JtKDFlNSwgMywgLjUpLAogIGJldGEgPSBybm9ybSgxZTUsIDAsIC4yKQopICU+JSAKICBnYXRoZXIgJT4lIAogIG11dGF0ZSh2YWx1ZSA9IGV4cCh2YWx1ZSkpICU+JSAKICBmaWx0ZXIoIWlzLm5hKHZhbHVlKSkgJT4lIAogIGdncGxvdChhZXMoeT1rZXksIHg9dmFsdWUpKSArCiAgICBnZW9tX2hhbGZleWVoKGZpbGwgPSBwYWxldHRlWzVdKSArCiAgICBjb29yZF9jYXJ0ZXNpYW4oeGxpbSA9IGMoMCwgNTApKSArCiAgICB0aGVtZV9idXJneWwoKSArIHlsYWIoIlByaW9yIikgKyB4bGFiKCJleHAocHJpb3IpIikKYGBgCgpgYGB7cn0KYjEwLjkgPC0KICBicm0oZGF0YSA9IGRmLCBmYW1pbHkgPSBwb2lzc29uLAogICAgICB0b3RhbF90b29scyB+IDEsCiAgICAgIHByaW9yID0gYyhwcmlvcihub3JtYWwoMywgLjUpLCBjbGFzcyA9IEludGVyY2VwdCkpLAogICAgICBjaGFpbnMgPSA0LCBjb3JlcyA9IDQsIHNhbXBsZV9wcmlvciA9ICJ5ZXMiLCByZWZyZXNoID0gMCkKYjEwLjkKYGBgCgoKYGBge3J9CmRmIDwtIGRmICU+JSAKICBtdXRhdGUobG9nX3BvcCA9IGxvZyhwb3B1bGF0aW9uKSkKYjEwLjEwIDwtCiAgYnJtKGRhdGEgPSBkZiwgZmFtaWx5ID0gcG9pc3NvbiwKICAgICAgdG90YWxfdG9vbHMgfiAwICsgY29udGFjdF9sb3cgKyBjb250YWN0X2hpZ2ggKyBsb2dfcG9wICsgY29udGFjdF9oaWdoOmxvZ19wb3AsCiAgICAgIHByaW9yID0gYyhwcmlvcihub3JtYWwoMCwgMSksIGNsYXNzID0gYikpLAogICAgICBjaGFpbnMgPSA0LCBjb3JlcyA9IDQsIHJlZnJlc2ggPSAwLCBzYW1wbGVfcHJpb3IgPSAieWVzIikKYjEwLjEwCmBgYAoKYGBge3J9Cmxvby4xMC4xMCA8LSBsb28oYjEwLjEwLCBzYXZlX3BzaXMgPSBUUlVFLCBjb3JlcyA9IDIpCnByaW50KGxvby4xMC4xMCkKcGxvdChsb28uMTAuMTAsIGxhYmVsX3BvaW50cyA9IFRSVUUpCmBgYAoKYGBge3J9CnRkIDwtIHRpYmJsZSgKICBwb3B1bGF0aW9uID0gc2VxKGZyb20gPSAwLCB0byA9IDI3NTAwMCwgbGVuZ3RoLm91dCA9IDI1MCkpICU+JSAKICBleHBhbmQobmVzdGluZyhwb3B1bGF0aW9uKSwgY29udGFjdCA9IGMoImxvdyIsICJoaWdoIikpICU+JSAKICBtdXRhdGUoCiAgICBsb2dfcG9wID0gbG9nKHBvcHVsYXRpb24pLAogICAgY29udGFjdF9oaWdoID0gaWZlbHNlKGNvbnRhY3QgPT0gImhpZ2giLCAxLCAwKSwKICAgIGNvbnRhY3RfbG93ID0gaWZlbHNlKGNvbnRhY3QgPT0gImxvdyIsIDAsIDEpKQoKdGQgPC0gdGQgJT4lCiAgYmluZF9jb2xzKAogICAgYXNfZGF0YV9mcmFtZSgKICAgICAgcHJlZGljdChiMTAuMTAsIG5ld2RhdGEgPSB0ZCkpKSAlPiUgCiAgZmlsdGVyKCFpcy5uYW4oRXN0aW1hdGUpKQoKcDEgPC0gdGQgJT4lIAogIGdncGxvdChhZXMoeCA9IHBvcHVsYXRpb24sIHkgPSBFc3RpbWF0ZSwgCiAgICAgICAgICAgICB5bWluID0gYFEyLjVgLCB5bWF4ID0gYFE5Ny41YCwKICAgICAgICAgICAgIGdyb3VwID0gY29udGFjdCkpICsKICBnZW9tX3JpYmJvbihjb2xvdXIgPSAidHJhbnNwYXJlbnQiLCBhbHBoYSA9IC42LAogICAgICAgICAgICAgIGFlcyhmaWxsID0gY29udGFjdCkpICsKICBnZW9tX2xpbmUoKSArCiAgZ2VvbV9wb2ludChkYXRhID0gZCwgaW5oZXJpdC5hZXMgPSBGQUxTRSwKICAgICAgICAgICAgIG1hcHBpbmcgPSBhZXMoeCA9IHBvcHVsYXRpb24sIHkgPSB0b3RhbF90b29scywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgIHNoYXBlID0gY29udGFjdCwgY29sb3VyID0gY29udGFjdCkpICsKICB0aGVtZV9idXJneWwoInRsIikgKyB5bGFiKCJQcmVkaWN0ZWQgVG90YWwgVG9vbHMiKSArCiAgc2NhbGVfZmlsbF9tYW51YWwodmFsdWVzID0gYyhwYWxldHRlWzNdLCBwYWxldHRlWzVdKSkgKwogIHNjYWxlX2NvbG9yX21hbnVhbCh2YWx1ZXMgPSBjKHBhbGV0dGVbNl0sIHBhbGV0dGVbN10pKSAjKwojICBnZ3RpdGxlKCJQb3N0ZXJpb3IgcHJlZGljdGlvbiBwbG90IiwKIyAgICAgICAgICAidG90YWxfdG9vbHMgfiBjb250YWN0X2xvdyArIGNvbnRhY3RfaGlnaCArIGxvZ19wb3AgKyBjb250YWN0X2hpZ2g6bG9nX3BvcCIpCnAyIDwtIHRkICU+JSAKICBnZ3Bsb3QoYWVzKHggPSBsb2dfcG9wLCB5ID0gRXN0aW1hdGUsIAogICAgICAgICAgICAgeW1pbiA9IGBRMi41YCwgeW1heCA9IGBROTcuNWAsCiAgICAgICAgICAgICBncm91cCA9IGNvbnRhY3QpKSArCiAgZ2VvbV9yaWJib24oY29sb3VyID0gInRyYW5zcGFyZW50IiwgYWxwaGEgPSAuNiwKICAgICAgICAgICAgICBhZXMoZmlsbCA9IGNvbnRhY3QpKSArCiAgZ2VvbV9saW5lKCkgKwogIGdlb21fcG9pbnQoZGF0YSA9IGQsIGluaGVyaXQuYWVzID0gRkFMU0UsCiAgICAgICAgICAgICBtYXBwaW5nID0gYWVzKHggPSBsb2cocG9wdWxhdGlvbiksIHkgPSB0b3RhbF90b29scywgCiAgICAgICAgICAgICAgICAgICAgICAgICAgIHNoYXBlID0gY29udGFjdCwgY29sb3VyID0gY29udGFjdCkpICsKICB0aGVtZV9idXJneWwoKSArIHlsYWIoIiIpICsgeGxhYigibG9nKHBvcHVsYXRpb24pIikgKwogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbj0ibm9uZSIpICsKICBzY2FsZV9maWxsX21hbnVhbCh2YWx1ZXMgPSBjKHBhbGV0dGVbM10sIHBhbGV0dGVbNV0pKSArCiAgc2NhbGVfY29sb3JfbWFudWFsKHZhbHVlcyA9IGMocGFsZXR0ZVs2XSwgcGFsZXR0ZVs3XSkpIAoKZ3JpZEV4dHJhOjpncmlkLmFycmFuZ2UocDEsIHAyLCBuY29sID0gMiwKICAgICAgICAgICAgICAgICAgICAgICAgdG9wID0gIlBvc3RlcmlvciBwcmVkaWN0aW9uIHBsb3QiKQpgYGAKCgojIyBFeGFtcGxlOiBFeHBvc3VyZSBhbmQgdGhlIG9mZnNldAoKVGhlIHJhdGUgcGFyYW1ldGVyLCAkXGxhbWJkYSQsIG9mIGEgUG9pc3NvbiBkaXN0cmlidXRpb24gY2FuIGJlIG5vcm1hbGl6ZWQgYWNyb3NzIGRpZmZlcmVudCBleHBvc3VyZSBmcmVxdWVuY2llcy4gTWNFbHJlYXRoIGdpdmVzIHRoZSBleGFtcGxlIG9mIGNvbWJpbmluZyBkYXRhIGZyb20gYSBkYWlseSBsb2cgYW5kIGEgd2Vla2x5IGxvZy4KCiQkCnlfaSBcc2ltIFBvaXNzb24oXGxhbWJkYV9pKQpcXApsb2coXGxhbWJkYV9pKSA9IGxvZyhcZnJhY3tcbXVfaX17XHRhdV9pfSkgPSBcYWxwaGEgKyBcYmV0YSB4X2kKXFwKbG9nKFxtdV9pKSA9IGxvZyhcdGF1X2kpICsgXGFscGhhICsgXGJldGEgeF9pCiQkCgokXHRhdSQgaXMganVzdCBhIGNvbHVtbiBvZiBkYXRhLCBvZnRlbiByZWZlcnJlZCB0byBhcyB0aGUgX29mZnNldF8uIExldHMgc2ltdWxhdGUgYW4gZXhhbXBsZSB0byBzaG93IGhvdyB0aGlzIHdvcmtzIHdoZXJlIG9uZSBtb25lc3RhcnkgcmVjb3JkcyBpdHMgbWFudXNjcmlwdCBjb21wbGV0aW9ucyBkYWlseSB3aGlsZSBhbm90aGVyIGRvZXMgc28gd2Vla2x5LgoKYGBge3J9Cm51bV9kYXlzIDwtIDMwCm51bV93ZWVrcyA8LSA0CgpkIDwtIGRhdGFfZnJhbWUoCiAgbWFudXNjcmlwdHMgPSBjKHJwb2lzKG51bV9kYXlzLCAxLjUpLAogICAgICAgICAgICAgICAgICBycG9pcyhudW1fd2Vla3MsIC41ICogNykpLAogIG51bV9kYXlzID0gYyhyZXAoMSwgMzApLCByZXAoNywgNCkpLAogIG1vbmFzdGVyeV9pZCA9IGMocmVwKDAsIDMwKSwgcmVwKDEsIDQpKSkKCgpkIDwtIGQgJT4lIAogIG11dGF0ZSgKICAgIGxvZ19udW1fZGF5cyA9IGxvZyhudW1fZGF5cyksCiAgICBtb25hc3RhcnlfMSA9IGFzLm51bWVyaWMobW9uYXN0ZXJ5X2lkID09IDApLAogICAgbW9uYXN0YXJ5XzIgPSBhcy5udW1lcmljKG1vbmFzdGVyeV9pZCA9PSAxKSkKCmQgJT4lIAogIHRhaWwoMTApCmBgYAoKTm93IGxldHMgZml0IHRoZSBtb2RlbC4KYGBge3J9CmIxMC4xNSA8LQogIGJybShkYXRhID0gZCwgZmFtaWx5ID0gcG9pc3NvbiwKICAgICAgbWFudXNjcmlwdHMgfiAwICsgb2Zmc2V0KGxvZ19udW1fZGF5cykgKyBtb25hc3RhcnlfMSArIG1vbmFzdGFyeV8yLAogICAgICBwcmlvciA9IGMocHJpb3Iobm9ybWFsKDAsIDEpLCBjbGFzcyA9IGIpKSwKICAgICAgc2FtcGxlX3ByaW9yID0gInllcyIsIHJlZnJlc2ggPSAwLAogICAgICBpdGVyID0gMjUwMCwgd2FybXVwID0gNTAwLCBjb3JlcyA9IDIsIGNoYWlucyA9IDIpCmIxMC4xNQpgYGAKCmBgYHtyfQpyYXRlcyA8LSBmaXhlZihiMTAuMTUpICU+JSAKICBhc19kYXRhX2ZyYW1lICU+JSAKICBwdWxsKEVzdGltYXRlKSAlPiUgCiAgZXhwCnN1YnRpdGxlIDwtIGdsdWUoIk1vbmFzdGFyeSAyIHByb2R1Y2VzIG1hbnVzY3JpcHRzIHtyb3VuZChyYXRlc1sxXSAvIHJhdGVzWzJdLCAxKX0gdGltZXMgYXMgZmFzdCBhcyBtb25hc3RhcnkgMSIpCgpwb3N0ZXJpb3Jfc2FtcGxlcyhiMTAuMTUpICU+JSAKICBzZWxlY3Qoc3RhcnRzX3dpdGgoImJfIikpICU+JSAKICB0cmFuc211dGUocmF0ZV9kaWZmID0gZXhwKGJfbW9uYXN0YXJ5XzEpIC8gZXhwKGJfbW9uYXN0YXJ5XzIpKSAlPiUKICBnYXRoZXIoUGFyYW1ldGVyLCBWYWx1ZSkgJT4lIAogIGdncGxvdChhZXMoeSA9IFBhcmFtZXRlciwgeCA9IFZhbHVlKSkgKwogICAgZ2VvbV92bGluZSh4aW50ZXJjZXB0ID0gcmF0ZXNbMV0gLyByYXRlc1syXSkgKwogICAgdGlkeWJheWVzOjpzdGF0X2ludGVydmFsaCgud2lkdGggPSBjKC45LCAuOTUsIC45OSkpICsKICAgIGdlb21fdmxpbmUoeGludGVyY2VwdCA9IDAsIGxpbmV0eXBlID0gMikgKwogICAgY29vcmRfY2FydGVzaWFuKHhsaW0gPSBjKDAsIDYpKSArCiAgICB0aGVtZV9idygpICsgdGhlbWUocGFuZWwuZ3JpZCA9IGVsZW1lbnRfYmxhbmsoKSwKICAgICAgICAgICAgICAgICAgICAgICBheGlzLnRleHQueSA9IGVsZW1lbnRfYmxhbmsoKSwKICAgICAgICAgICAgICAgICAgICAgICBheGlzLnRpY2tzLnkgPSBlbGVtZW50X2JsYW5rKCksCiAgICAgICAgICAgICAgICAgICAgICAgbGVnZW5kLnBvc2l0aW9uID0gYygxLCAxKSwKICAgICAgICAgICAgICAgICAgICAgICBsZWdlbmQuanVzdGlmaWNhdGlvbiA9IGMoMSwgMSksCiAgICAgICAgICAgICAgICAgICAgICAgbGVnZW5kLmJhY2tncm91bmQgPSBlbGVtZW50X3JlY3QoZmlsbCA9ICJ0cmFuc3BhcmVudCIpKSArCiAgICB5bGFiKCIiKSArIHhsYWIoIlRpbWVzIGluY3JlYXNlIGluIG1hbnVzY3JpcHQgY3JlYXRpb24gcmF0ZSIpICsKICAgIGFubm90YXRlKCJ0ZXh0IiwgeCA9IDMuMSwgeSA9IDEuNCwKICAgICAgICAgICAgIGxhYmVsID0gZ2x1ZTo6Z2x1ZSgKICAgICAgICAgICAgICAgIk1lZGlhbiBIRFBJOiB7cm91bmQocmF0ZXNbMV0gLyByYXRlc1syXSwgMSl9IikpICsKICAgIGdndGl0bGUoIk1vbmFzdGFyeSAyIGNyZWF0ZXMgbWFudXNjcmlwdHMgc2lnbmlmaWNhbnRseSBmYXN0ZXIgdGhhbiBNb25hc3RhcnkgMSIpCmBgYAoKIyMgTXVsdGlub21pYWwgaW4gZGlzZ3Vpc2UgYXMgUG9pc3NvbgpgYGB7cn0Kc3VwcHJlc3NQYWNrYWdlU3RhcnR1cE1lc3NhZ2VzKGxpYnJhcnkocmV0aGlua2luZykpCmRhdGEoVUNCYWRtaXQpCmRmIDwtIGFzX2RhdGFfZnJhbWUoVUNCYWRtaXQpCmRldGFjaCgicGFja2FnZTpyZXRoaW5raW5nIikKZGYKYGBgCgpgYGB7cn0KYi5iaW5vbSA8LSBicm0oYWRtaXQgfCB0cmlhbHMoYXBwbGljYXRpb25zKSB+IDEsCiAgICAgICAgICAgICAgIGRmLCBiaW5vbWlhbCwKICAgICAgICAgICAgICAgcHJpb3IgPSBwcmlvcihub3JtYWwoMCwgMTAwKSwgY2xhc3MgPSAiSW50ZXJjZXB0IiksCiAgICAgICAgICAgICAgIHJlZnJlc2ggPSAwKQpiLmJpbm9tCmBgYAoKCmBgYHtyfQpkZiRyZWogPC0gZGYkcmVqZWN0ICMgJ3JlamVjdCcgaXMgYSByZXNlcnZlZCB3b3JkCm1fcG9pcyA8LSBtYXAyc3RhbigKICAgIGFsaXN0KAogICAgICAgIGFkbWl0IH4gZHBvaXMobGFtYmRhMSksCiAgICAgICAgcmVqIH4gZHBvaXMobGFtYmRhMiksCiAgICAgICAgbG9nKGxhbWJkYTEpIDwtIGExLAogICAgICAgIGxvZyhsYW1iZGEyKSA8LSBhMiwKICAgICAgICBjKGExLGEyKSB+IGRub3JtKDAsMTAwKQogICAgKSwKICAgIGRhdGE9ZGYsICBjaGFpbnM9MyAsIGNvcmVzPTMgKQpzdW1tYXJ5KG1fcG9pcykKYGBgCgpgYGB7cn0KayA8LSBhcy5udW1lcmljKGNvZWYobV9wb2lzKSkKdGliYmxlKAogIGJpbm9tX2ZpdCA9IGludl9sb2dpdF9zY2FsZWQoZml4ZWYoYi5iaW5vbSkpWywxXSwKICBwb2lzX2ZpdCA9IGV4cChrWzFdKSAvIChleHAoa1sxXSkgKyBleHAoa1syXSkpKQpgYGAKCiMgQ2Vuc29yaW5nIGFuZCBzdXJ2aXZhbAoKU3Vydml2YWwgbW9kZWxzIG1lYXN1cmUgX2Rpc3BsYWNlbWVudF8sIHdoaWNoIGFyZSBjb250aW51b3VzIGRldmlhdGlvbnMgZnJvbSBzb21lIHBvaW50IG9mIHJlZmVyZW5jZS4gRXhhbXBsZXMgb2YgdGhpcyBhcmUgZHVyYXRpb25zICh0aW1lIHRvIGRpc2NoYXJnZSkuIFdoZW4gYWxsIHdlIGtub3cgaXMgdGhlIGF2ZXJhZ2UgZGlzcGxhY2VtZW50LCB0aGUgZXhwb25lbnRpYWwgZGlzZ3JpYnV0aW9uIGlzIHRoZSBtYXhpbXVtIGVudHJvcHkgZGlzdHJpYnV0aW9uLiBBbHRlcm5hdGl2ZWx5LCB0aGUgZ2FtbWEgZGlzdHJpYnV0aW9uIGlzIHRoZSBtYXhpbXVtIGVudHJvcHkgZGlzdHJpYnV0aW9uIGZvciBmaXhlZCBtZWFuIHZhbHVlIGFuZCBmaXhlZCBtZWFuIG1hZ25pdHVkZS4gCgpTdXZpdmFsIG1vZGVscyBhbHNvIGRlYWwgd2l0aCBfY2Vuc29yaW5nXywgd2hpY2ggaXMgd2hlbiBzb21lIG90aGVyIGV2ZW50IGhhcHBlbnMgYW5kIGdldHMgaW4gdGhlIHdheSBvZiBtZWFzdXJpbmcgdGhlIGV2ZW50IG9mIGludGVyZXN0IChkZWF0aCBpbnN0ZWFkIG9mIGRpc2NoYXJnZSkuCgojIyBDYXRzIGV4YW1wbGUKCkxldHMgZXhhbWluZSBjYXRzIGluIEF1c3RpbiB3aGVyZSB3ZSBhcmUgaW50ZXJlc3RlZCBpbiB0aGUgdGltZSB1bnRpbCB0aGVpciBhZG9wdGlvbi4gV2UgaGF2ZSBvbmUgZW50cnkgcGVyIGNhdCBpbiB0aGUgZGF0YSBzZXQuCgpgYGB7cn0Kc3VwcHJlc3NQYWNrYWdlU3RhcnR1cE1lc3NhZ2VzKGxpYnJhcnkocmV0aGlua2luZykpCmRhdGEoQXVzdGluQ2F0cykKZCA8LSBhc19kYXRhX2ZyYW1lKEF1c3RpbkNhdHMpICU+JSAKICBtdXRhdGUoYWRvcHQgPSBpZmVsc2Uob3V0X2V2ZW50ID09ICJBZG9wdGlvbiIsIDFMLCAwTCkpCnJtKEF1c3RpbkNhdHMpCmRldGFjaCgicGFja2FnZTpyZXRoaW5raW5nIikKKGQgJT4lIAogIHRyYW5zbXV0ZShpZCwKICAgICAgICAgICAgYWRvcHQsIAogICAgICAgICAgICBkYXlzX3RvX2V2ZW50LCAKICAgICAgICAgICAgaXNfYmxhY2sgPSBhcy5pbnRlZ2VyKGNvbG9yPT0iQmxhY2siKSkgLT4KICBkKQpgYGAKCldlJ2xsIHJ1biB0aGUgZm9sbG93aW5nIG1vZGVsIHRvIHByZWRpY3QgdGhlIG51bWJlciBvZiBkYXlzLiBGb3IgYWRvcHRlZCBjYXRzLCB0aGUgZGF5cyB0byBhZG9wdGlvbiBpcyBleHBvbmVudGlhbGx5IGRpc3RyaWJ1dGVkIGFuZCBlYXN5IHRvIGVuY29kZS4gRm9yIG5vbi1hZG9wdGVkIGNhdHMsIHdlIGVtcGxveSB0aGUgY3VtdWxhdGl2ZSBwcm9iYWJpbGl0eSBkaXN0cmlidXRpb24gd2hpY2ggZ2l2ZXMgdGhlIHByb3BvcnRpb24gb2YgY2F0cyBhZG9wdGVkIGJlZm9yZSBvciBhdCBhIGNlcnRhaW4gbnVtYmVyIG9mIGRheXMuIFRoZXJlZm9yZSwgJDEgLSBjdW11bGF0aXZlXF9kaXN0cmlidXRpb24kIGdpdmVzIHRoZSBwcm9wb3J0aW9uIG9mIGNhdHMgbm90IHlldCBhZG9wdGVkIGF0IGRheSAkZCQuIFRoaXMgaXMgY2FsbGVkIHRoZSBjb21wbGVtZW50YXJ5IGN1bXVsYXRpdmUgcHJvYmFiaWxpdHkgZGlzdHJpYnV0aW9uIGFuZCBpcyBkaXNwbGF5ZWQgaW4gdGhlIHNlY29uZCBsaW5lIGJlbG93LiAKCiQkCkRfaSB8IEFfaSA9IDEgXHNpbSBFeHBvbmVudGlhbChcbGFtYmRhX2kpClxcCkRfaSB8IEFfaSA9IDAgXHNpbSBFeHBvbmVudGlhbENDREYoXGxhbWJkYV9pKQpcXApcbGFtYmRhX2kgPSBcZnJhY3sxfXtcbXVfaX0KXFwKbG9nIFxtdV9pID0gXGFscGhhX3tDSURbaV19CiQkCgoKTGV0cyBydW4gdGhpcyBpbiBTVEFOLgoKYGBge3J9Cm0xMS4xNCA8LSB1bGFtKAogIGFsaXN0KAogICAgZGF5c190b19ldmVudCB8IGFkb3B0ZWQgPT0gMSB+IGV4cG9uZW50aWFsKGxhbWJkYSksCiAgICBkYXlzX3RvX2V2ZW50IHwgYWRvcHRlZCA9PSAwIH4gY3VzdG9tKGV4cG9uZW50aWFsX2xjY2RmKCFZIHwgbGFtYmRhKSksCiAgICBsYW1iZGEgPC0gMS4wIC8gbXUsCiAgICAjIEZpdCB0byBsb2cgbXUgZm9yIG51bWVyaWMgc3RhYmlsaXR5CiAgICBsb2cobXUpIDwtIGFbY29sb3JfaWRdLAogICAgYVtjb2xvcl9pZF0gfiBub3JtYWwoMCwgMSkKICApLCAKICBkYXRhID0gbGlzdCgKICAgIGRheXNfdG9fZXZlbnQgPSBhcy5udW1lcmljKGQkZGF5c190b19ldmVudCksCiAgICBjb2xvcl9pZCA9IGlmZWxzZShkJGNvbG9yPT0iQmxhY2siLCAxTCwgMkwpLAogICAgYWRvcHRlZCA9IGQkYWRvcHQpLAogIGNoYWlucyA9IDQsIGNvcmVzID0gNCkKCnByZWNpcyhtMTEuMTQsIDIpCmBgYAoKCmBgYHtyfQpwb3N0IDwtIGFzX2RhdGFfZnJhbWUoZXh0cmFjdC5zYW1wbGVzKG0xMS4xNCkkYSkgJT4lIAogIHJlbmFtZShibGFjayA9IFYxLAogICAgICAgICBvdGhlciA9IFYyKQpwb3N0ICU+JSAKICBnYXRoZXIoY29sb3VyKSAlPiUgCiAgbXV0YXRlKHZhbHVlID0gZXhwKHZhbHVlKSkgJT4lIAogIGdncGxvdChhZXMoeCA9IHZhbHVlLCB5ID0gY29sb3VyKSkgKwogICAgZ2VvbV9oYWxmZXllaCgud2lkdGggPSBjKC45NSwgLjUpKSArCiAgICB0aGVtZShwYW5lbC5iYWNrZ3JvdW5kID0gZWxlbWVudF9yZWN0KGZpbGwgICA9ICJ0cmFuc3BhcmVudCIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbG91ciA9ICJibGFjayIpKSArCiAgICB4bGFiKCJEYXlzIHRvIGFkb3B0aW9uIikgKyB5bGFiKCJDYXQgY29sb3VyIikKYGBgCgoKCiMgSG9tZXdvcmsKCiMjIDEwSDMKCkZpcnN0IGxldHMgZ2F0aGVyIGFuZCBwcmVwcm9jZXNzIHRoZSBkYXRhLgoKYGBge3J9CmxpYnJhcnkoTUFTUykKZGF0YShlYWdsZXMpCmRmIDwtIGFzX2RhdGFfZnJhbWUoZWFnbGVzKQpybShlYWdsZXMpCmRldGFjaCgicGFja2FnZTpNQVNTIikKZGYgPC0gZGYgJT4lIAogIHRyYW5zbXV0ZSgKICAgIHN1Y2Nlc3NlcyA9IHksCiAgICBhdHRlbXB0cyA9IG4sCiAgICBwaXJhdGVfc2l6ZSA9IGlmZWxzZShQID09ICdMJywgImxhcmdlIiwgInNtYWxsIiksCiAgICBhZHVsdCA9IGlmZWxzZShBID09ICJJIiwgImltbWF0dXJlIiwgImFkdWx0IiksCiAgICB2aWN0aW1fc2l6ZSA9IGlmZWxzZShWID09ICJMIiwgImxhcmdlIiwgInNtYWxsIikpCgpsaWJyYXJ5KHJlY2lwZXMpCgpkZiA8LSBkZiAlPiUgCiAgcmVjaXBlKHN1Y2Nlc3NlcyB+IC4pICU+JSAKICBzdGVwX2R1bW15KHBpcmF0ZV9zaXplLCBhZHVsdCwgdmljdGltX3NpemUsIG9uZV9ob3QgPSBUUlVFKSAlPiUgCiAgcHJlcCAlPiUgCiAganVpY2UKCmRmCmBgYAoKV2UgZHJvcCB0aGUgaW50ZXJjZXB0IGFuZCBpbmNsdWRlIGFsbCBmYWN0b3IgbGV2ZWxzIG9mIG91ciBiaW5hcnkgdmFyaWFibGVzLgpgYGB7cn0KaHlwb3RoZXNpcyA8LSBiZihzdWNjZXNzZXMgfCB0cmlhbHMoYXR0ZW1wdHMpIH4gMCArIC4pCmdldF9wcmlvcihoeXBvdGhlc2lzLCBmYW1pbHkgPSBiaW5vbWlhbCwgZGF0YSA9IGRmKQpgYGAKCk91ciBgc3RhbmAgZXN0aW1hdGVzIGxvb2sgc29saWQgc28gbGV0cyBqdXN0IHJ1biB3aXRoIHRob3NlLgpgYGB7cn0KZml0LjEwaDMgPC0gYnJtKGZhbWlseSA9IGJpbm9taWFsLCBkYXRhID0gZGYsCiAgICAgICAgICAgICAgICBmb3JtdWxhID0gaHlwb3RoZXNpcywKICAgICAgICAgICAgICAgIHByaW9yID0gYyhwcmlvcihub3JtYWwoMCwgNCksIGNsYXNzID0gYikpLAogICAgICAgICAgICAgICAgY2hhaW5zID0gNCwgY29yZXMgPSA0LCByZWZyZXNoID0gMCkKcGxvdChmaXQuMTBoMykKYGBgCgpMZXRzIHRha2UgYSBsb29rIGF0IG91ciBlc3RpbWF0ZXMgb24gdGhlIG91dGNvbWUgc2NhbGUuCmBgYHtyfQpwb3N0ZXJpb3Jfc2FtcGxlcyhmaXQuMTBoMykgJT4lIAogIHNlbGVjdChzdGFydHNfd2l0aCgiYl8iKSkgJT4lIAogIGdhdGhlcihwYXJhbWV0ZXIsIHZhbHVlKSAlPiUgCiAgbXV0YXRlKHZhbHVlID0gaW52X2xvZ2l0X3NjYWxlZCh2YWx1ZSkpICU+JQogIGdncGxvdChhZXMoeT1wYXJhbWV0ZXIsIHggPSB2YWx1ZSkpICsKICAgIGdlb21faGFsZmV5ZWgoLndpZHRoID0gYyguNSwgLjk1KSkKYGBgCgoKCgoKYGBge3J9CmFzX2RhdGFfZnJhbWUocHJlZGljdChmaXQuMTBoMywgbmV3ZGF0YSA9IGRmKSkgJT4lIAogIGJpbmRfY29scyhpbmRleD0xOm5yb3coZGYpLCBkZiwgLikgJT4lIAogIGdncGxvdChhZXMoeCA9IGluZGV4LCBjb2xvdXIgPSBhcy5mYWN0b3IocGlyYXRlX3NpemVfbGFyZ2UpLCBzaGFwZSA9IGFzLmNoYXJhY3RlcihhZHVsdF9hZHVsdCkpKSArCiAgICBnZW9tX2Nyb3NzYmFyKGFlcyh5ID0gRXN0aW1hdGUvYXR0ZW1wdHMsIHltaW4gPSBgUTIuNWAvYXR0ZW1wdHMsIHltYXggPSBgUTk3LjVgL2F0dGVtcHRzKSkgKwogICAgZ2VvbV9wb2ludChhZXMoeSA9IHN1Y2Nlc3Nlcy9hdHRlbXB0cyksIHNpemUgPSAzKSArCiAgICB0aGVtZShwYW5lbC5ncmlkID0gZWxlbWVudF9ibGFuaygpKQpgYGAKCmBgYHtyfQpmaXQuMTBoMy4yIDwtIHVwZGF0ZShmaXQuMTBoMywgcmVmcmVzaCA9IDAsCiAgICAgICAgICAgICAgICAgICAgIGZvcm11bGEgPSBiZihzdWNjZXNzZXMgfCB0cmlhbHMoYXR0ZW1wdHMpIH4gMSArIGFkdWx0X2FkdWx0ICsKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBpcmF0ZV9zaXplX2xhcmdlICsKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHZpY3RpbV9zaXplX2xhcmdlICsKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGFkdWx0X2FkdWx0OnBpcmF0ZV9zaXplX2xhcmdlKSkKZml0LjEwaDMuMgoKYGBgCgpgYGB7cn0KcGxvdChmaXQuMTBoMy4yKQpgYGAKCgpgYGB7cn0Kd2FpYyhmaXQuMTBoMywgZml0LjEwaDMuMikKYGBgCmBgYHtyfQpsb28oZml0LjEwaDMsIGZpdC4xMGgzLjIsIHJlbG9vID0gVFJVRSkKYGBgCgoKIyMgMTBILjQKCkZpcnN0IGxldHMgbG9hZCB0aGUgZGF0YSBhbmQgdGFrZSBhIGxvb2sgYXQgaXQuCmBgYHtyIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9CmxpYnJhcnkocmV0aGlua2luZykKZGF0YShzYWxhbWFuZGVycykKZGYgPC0gYXNfZGF0YV9mcmFtZShzYWxhbWFuZGVycykKcm0oc2FsYW1hbmRlcnMpCmRldGFjaCgicGFja2FnZTpyZXRoaW5raW5nIikKCmRmIDwtIGRmICU+JSAKICB0cmFuc211dGUoCiAgICBzaXRlID0gU0lURSwKICAgIG51bV9zYWxhbWFuID0gU0FMQU1BTiwKICAgIHBjdF9jb3ZlciA9IFBDVENPVkVSLAogICAgYWdlX3RyZWVzID0gRk9SRVNUQUdFKQoKZGYgJT4lIHNraW1yOjpza2ltKCkKYGBgCgpOb3cgZm9yIHNvbWUgcHJlcHJvY2Vzc2luZy4gV2Ugd2FudCB0byB0cnkgYSBmZXcgZGlmZmVyZW50IHRyYW5zZm9ybWF0aW9ucyBvZiBvdXIgcHJlZGljdG9ycy4KYGBge3J9CmRtIDwtIGRmICU+JSAKICBtdXRhdGUobG9nX2FnZV90cmVlcyA9IGxvZyhhZ2VfdHJlZXMpKSAlPiUgCiAgZmlsdGVyKGlzLmZpbml0ZShsb2dfYWdlX3RyZWVzKSkgJT4lIAogIG11dGF0ZSgKICAgIHBjdF9jb3Zlcl9jcyA9IHNjYWxlKHBjdF9jb3ZlciksCiAgICBsb2dfYWdlX3RyZWVzX2NzID0gc2NhbGUobG9nX2FnZV90cmVlcyksCiAgICBhZ2VfdHJlZXNfY3MgPSBzY2FsZShhZ2VfdHJlZXMpKQoKZG0gJT4lIGhlYWQKYGBgCgpJbnRlcmVzdGluZ2x5LCB0aGUgYmVzdCBwZXJmb3JtaW5nIG1vZGVsIG9ubHkgdXNlcyB0aGUgcGVyY2VudCBvZiBjb3ZlcmFnZS4gSXQgYXBwZWFycyB0aGUgYWdlIG9mIHRyZWVzIGFkZHMgdG9vIG11Y2ggbm9pc2UgYW5kIGRvZXMgbm90IGdlbmVyYWxpemUgd2VsbC4KYGBge3J9CmZpdC4xMGguNC4xIDwtIGJybShudW1fc2FsYW1hbiB+IDEsIHJlZnJlc2ggPSAwLAogICAgICAgICAgICAgICAgIGZhbWlseSA9IHBvaXNzb24oKSwgZGF0YSA9IGRtLAogICAgICAgICAgICAgICAgIHByaW9yID0gYygKICAgICAgICAgICAgICAgICAgIHByaW9yKG5vcm1hbCgwLCAxKSwgY2xhc3MgPSAiSW50ZXJjZXB0IikpLAogICAgICAgICAgICAgICAgIHNhbXBsZV9wcmlvciA9ICJ5ZXMiLCByZWZyZXNoID0gMCwKICAgICAgICAgICAgICAgICBjaGFpbnMgPSA0LCBjb3JlcyA9IDQpCgpmaXQuMTBoLjQuMiA8LSB1cGRhdGUoZml0LjEwaC40LjEsIG5ld2RhdGEgPSBkbSwgcmVmcmVzaCA9IDAsCiAgICAgICAgICAgICAgICAgICAgICBudW1fc2FsYW1hbiB+IDEgKyBwY3RfY292ZXJfY3MsCiAgICAgICAgICAgICAgICAgICAgICBwcmlvciA9IGMoCiAgICAgICAgICAgICAgICAgICAgICAgIHByaW9yKG5vcm1hbCgwLCAxKSwgY2xhc3MgPSAiSW50ZXJjZXB0IiksCiAgICAgICAgICAgICAgICAgICAgICAgIHByaW9yKG5vcm1hbCgwLCAuNSksIGNsYXNzID0gImIiKSkpCgpmaXQuMTBoLjQuMyA8LSB1cGRhdGUoZml0LjEwaC40LjIsIG5ld2RhdGEgPSBkbSwgcmVmcmVzaCA9IDAsCiAgICAgICAgICAgICAgICAgICAgICBudW1fc2FsYW1hbiB+IDEgKyBwY3RfY292ZXJfY3MgKyBhZ2VfdHJlZXNfY3MpCgpmaXQuMTBoLjQuNCA8LSB1cGRhdGUoZml0LjEwaC40LjMsIG5ld2RhdGEgPSBkbSwgcmVmcmVzaCA9IDAsCiAgICAgICAgICAgICAgICAgICAgICBudW1fc2FsYW1hbiB+IDEgKyBwY3RfY292ZXJfY3MgKyBsb2dfYWdlX3RyZWVzX2NzKQoKbG9vKGZpdC4xMGguNC4xLCBmaXQuMTBoLjQuMiwgZml0LjEwaC40LjMsIGZpdC4xMGguNC40LAogICAgcmVsb28gPSBUUlVFKQpgYGAKCldlIGNhbiBpbnRlcnByZXQgdGhlIHdpbm5pbmcgbW9kZWwgbGlrZSBzbzoKCjEuIFdoZW4gdGhlIHBlcmNlbnQgb2YgY292ZXJhZ2UgaXMgYXQgaXRzIGF2ZXJhZ2UgdmFsdWUgKDYwLjMpLCB3ZSdsbCBmaW5kIDEuNyBzYWxhbWFuZGVycyBwZXIgcGxvdC4KMi4gQSBzaW5nbGUgc3RhbmRhcmQgZGV2aWF0aW9uIGNoYW5nZSBpbiBjb3ZlcmFnZSAoKy8tIDM1LjEpIHdpbGwgY2F1c2UgYSAyLjc2IGluY3JlYXNlIGluIHRoZSByYXRlIG9mIHNhbGFtYW5kZXJzIGZvdW5kIHBlciBwbG90Lm4KCmBgYHtyfQpmaXhlZihmaXQuMTBoLjQuMilbLDFdICU+JSAKICBleHAKCnBvc3Rlcmlvcl9zYW1wbGVzKGZpdC4xMGguNC4yKSAlPiUgCiAgc2VsZWN0KHN0YXJ0c193aXRoKCJiXyIpLCBzdGFydHNfd2l0aCgicHJpb3IiKSkgJT4lIAogIGdhdGhlcihwYXJhbWV0ZXIsIHZhbHVlKSAlPiUgCiAgbXV0YXRlKHZhbHVlID0gZXhwKHZhbHVlKSkgJT4lIAogIGdncGxvdChhZXMoeT1wYXJhbWV0ZXIsIHggPSB2YWx1ZSkpICsKICAgIGdlb21faGFsZmV5ZWgoKSArIAogICAgdGhlbWVfYncoKSArCiAgICBnZ3RpdGxlKCJQb3N0ZXJpb3IgZGlzdHJpYnV0aW9uIGZvciBlZmZlY3QgZXN0aW1hdGVzIikKYGBgCgojIyBGcmVxdWVudGlzdCBhcHByb2FjaAoKSnVzdCB0byBndXQgY2hlY2sgbXkgYmF5ZXNpYW4gZmluZGluZ3MsIGxldHMgcHJvdmUgdGhhdCB0aGUgc2ltcGxlciBtb2RlbCB3b3JrcyAKIApgYGB7cn0KbGlicmFyeShyc2FtcGxlKQpsaWJyYXJ5KHlhcmRzdGljaykKCmZvcm11bGEuaW50ICAgICAgICAgICAgICAgIDwtIGZvcm11bGEobnVtX3NhbGFtYW4gfiAxKQpmb3JtdWxhLmNvdmVyLmNzICAgICAgICAgICA8LSBmb3JtdWxhKG51bV9zYWxhbWFuIH4gMSArIHBjdF9jb3Zlcl9jcykKZm9ybXVsYS5jb3Zlci50cmVlcyAgICAgICAgPC0gZm9ybXVsYShudW1fc2FsYW1hbiB+IDEgKyBwY3RfY292ZXJfY3MgKyBhZ2VfdHJlZXMpCmZvcm11bGEuY292ZXIubG9ndHJlZXMgICAgIDwtIGZvcm11bGEobnVtX3NhbGFtYW4gfiAxICsgcGN0X2NvdmVyX2NzICsgbG9nX2FnZV90cmVlcykKZm9ybXVsYS5jb3ZlcmNzLmxvZ3RyZWVzY3MgPC0gZm9ybXVsYShudW1fc2FsYW1hbiB+IDEgKyBwY3RfY292ZXJfY3MgKyBsb2dfYWdlX3RyZWVzX2NzKQoKZml0X21vZGVsIDwtIGZ1bmN0aW9uKHNwbGl0cywgZm9ybXVsYSkgewogIGZpdC5ub25lIDwtIGdsbShmb3JtdWxhLCAKICAgICAgICAgICAgICAgICAgZmFtaWx5ID0gcG9pc3NvbigpLCBkYXRhID0gYW5hbHlzaXMoc3BsaXRzKSkKICAKICBob2xkb3V0IDwtIGFzc2Vzc21lbnQoc3BsaXRzKQogIHByZWRzIDwtIHByZWRpY3QoZml0Lm5vbmUsIG5ld2RhdGEgPSBob2xkb3V0KQogIAogIHJtc2VfdmVjKGhvbGRvdXQkbnVtX3NhbGFtYW4sIHByZWRzKQp9Cgpib290c3RyYXBzKGRtLCB0aW1lcyA9IDEwMDApICU+JQogIG11dGF0ZSgKICAgIGludGVyY2VwdCAgICAgICAgICA9IG1hcF9kYmwoc3BsaXRzLCB+IGZpdF9tb2RlbCgueCwgZm9ybXVsYS5pbnQpKSwKICAgIGNvdmVyX2NzICAgICAgICAgICA9IG1hcF9kYmwoc3BsaXRzLCB+IGZpdF9tb2RlbCgueCwgZm9ybXVsYS5jb3Zlci5jcykpLAogICAgY292ZXJfdHJlZXMgICAgICAgID0gbWFwX2RibChzcGxpdHMsIH4gZml0X21vZGVsKC54LCBmb3JtdWxhLmNvdmVyLnRyZWVzKSksCiAgICBjb3Zlcl9sb2d0cmVlcyAgICAgPSBtYXBfZGJsKHNwbGl0cywgfiBmaXRfbW9kZWwoLngsIGZvcm11bGEuY292ZXIubG9ndHJlZXMpKSwgCiAgICBjb3ZlcmNzX2xvZ3RyZWVzY3MgPSBtYXBfZGJsKHNwbGl0cywgfiBmaXRfbW9kZWwoLngsIGZvcm11bGEuY292ZXJjcy5sb2d0cmVlc2NzKSkpICU+JSAKICBhcy5kYXRhLmZyYW1lICU+JSAKICBzZWxlY3QoLXNwbGl0cywgLWlkKSAlPiUgCiAgZ2F0aGVyKG1vZGVsLCBybXNlKSAlPiUgCiAgZ3JvdXBfYnkobW9kZWwpICU+JSAKICBtZWFuX2hkaSgpICU+JSAKICBhcnJhbmdlKHJtc2UpCmBgYAoKCgoKCgoKCgoKCgo=